Compare commits
29 Commits
c50098a55c
...
codex/enha
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
96989d0ec3 | ||
|
|
12a82372f4 | ||
|
|
efa57c4ba8 | ||
|
|
3f9be316ce | ||
|
|
385a842ab8 | ||
|
|
320f8c4266 | ||
|
|
12b0a0881e | ||
|
|
dc18b08a3a | ||
|
|
df183230d8 | ||
|
|
baf3997fb6 | ||
|
|
a2d48f6a03 | ||
|
|
fb2533fa4c | ||
|
|
12304d5890 | ||
|
|
a0bdb09c03 | ||
|
|
c89dc04d4c | ||
|
|
7a175923b0 | ||
|
|
ed48623f27 | ||
|
|
228c3fb704 | ||
|
|
5f14a4050a | ||
|
|
eb1619c56c | ||
|
|
9226442525 | ||
|
|
49a571215d | ||
|
|
0e2f3dddbc | ||
|
|
f68321c802 | ||
|
|
20fc90a3fb | ||
|
|
51c82d3d95 | ||
|
|
a4577ddcf3 | ||
|
|
5f3fcef1ec | ||
|
|
16907aadf8 |
8
.env.api
8
.env.api
@@ -5,6 +5,14 @@ DATABASE_URL="mysql://platform:platform@db:3306/platform"
|
||||
CENTRAL_DATABASE_URL="mysql://root:asjdnfqTash37faggT@db:3306/central_platform"
|
||||
REDIS_URL="redis://redis:6379"
|
||||
|
||||
# Meilisearch (optional)
|
||||
MEILI_HOST="http://meilisearch:7700"
|
||||
MEILI_API_KEY="dev-meili-master-key"
|
||||
MEILI_INDEX_PREFIX="tenant_"
|
||||
|
||||
# JWT, multi-tenant hints, etc.
|
||||
JWT_SECRET="devsecret"
|
||||
TENANCY_STRATEGY="single-db"
|
||||
|
||||
|
||||
CENTRAL_SUBDOMAINS="central,admin"
|
||||
|
||||
4
.env.web
4
.env.web
@@ -1,5 +1,5 @@
|
||||
NUXT_PORT=3001
|
||||
NUXT_HOST=0.0.0.0
|
||||
|
||||
# Point Nuxt to the API container (not localhost)
|
||||
NUXT_PUBLIC_API_BASE_URL=http://jupiter.routebox.co:3000
|
||||
# Nitro BFF backend URL (server-only, not exposed to client)
|
||||
BACKEND_URL=https://backend.routebox.co
|
||||
83
DEBUG_INCOMING_CALL.md
Normal file
83
DEBUG_INCOMING_CALL.md
Normal file
@@ -0,0 +1,83 @@
|
||||
# Debugging Incoming Call Issue
|
||||
|
||||
## Current Problem
|
||||
- Hear "Connecting to your call" message (TwiML is executing)
|
||||
- No ring on mobile after "Connecting" message
|
||||
- Click Accept button does nothing
|
||||
- Call never connects
|
||||
|
||||
## Root Cause Hypothesis
|
||||
The Twilio Device SDK is likely **NOT receiving the incoming call event** from Twilio's Signaling Server. This could be because:
|
||||
|
||||
1. **Identity Mismatch**: The Device's identity (from JWT token) doesn't match the `<Client>ID</Client>` in TwiML
|
||||
2. **Device Not Registered**: Device registration isn't completing before the call arrives
|
||||
3. **Twilio Signaling Issue**: Device isn't connected to Twilio Signaling Server
|
||||
|
||||
## How to Debug
|
||||
|
||||
### Step 1: Check Device Identity in Console
|
||||
When you open the softphone dialog, **open Browser DevTools Console (F12)**
|
||||
|
||||
You should see logs like:
|
||||
```
|
||||
Token received, creating Device...
|
||||
Token identity: e6d45fa3-a108-4085-81e5-a8e05e85e6fb
|
||||
Token grants: {voice: {...}}
|
||||
Registering Twilio Device...
|
||||
✓ Twilio Device registered - ready to receive calls
|
||||
Device identity: e6d45fa3-a108-4085-81e5-a8e05e85e6fb
|
||||
Device state: ready
|
||||
```
|
||||
|
||||
**Note the Device identity value** - e.g., "e6d45fa3-a108-4085-81e5-a8e05e85e6fb"
|
||||
|
||||
### Step 2: Check Backend Logs
|
||||
When you make an inbound call, look for backend logs showing:
|
||||
|
||||
```
|
||||
╔════════════════════════════════════════╗
|
||||
║ === INBOUND CALL RECEIVED ===
|
||||
╚════════════════════════════════════════╝
|
||||
...
|
||||
Client IDs to dial: e6d45fa3-a108-4085-81e5-a8e05e85e6fb
|
||||
First Client ID format check: "e6d45fa3-a108-4085-81e5-a8e05e85e6fb" (length: 36)
|
||||
```
|
||||
|
||||
### Step 3: Compare Identities
|
||||
The Device identity from frontend console MUST MATCH the Client ID from backend logs.
|
||||
|
||||
**If they match**: The issue is with Twilio Signaling or Device SDK configuration
|
||||
**If they don't match**: We found the bug - identity mismatch
|
||||
|
||||
### Step 4: Monitor Incoming Event
|
||||
When you make the inbound call, keep watching the browser console for:
|
||||
|
||||
```
|
||||
🔔 Twilio Device INCOMING event received: {...}
|
||||
```
|
||||
|
||||
**If this appears**: The Device SDK IS receiving the call, so the Accept button issue is frontend
|
||||
**If this doesn't appear**: The Device SDK is NOT receiving the call, so it's an identity/registration issue
|
||||
|
||||
## What Changed
|
||||
- Frontend now relies on **Twilio Device SDK `incoming` event** (not Socket.IO) for showing incoming call
|
||||
- Added comprehensive logging to Device initialization
|
||||
- Added logging to Accept button handler
|
||||
- Backend logs Device ID format for comparison
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Make an inbound call
|
||||
2. Check browser console for the 5 logs above
|
||||
3. Check backend logs for Client ID
|
||||
4. Look for "🔔 Twilio Device INCOMING event" in browser console
|
||||
5. Try clicking Accept and watch console for "📞 Accepting call" logs
|
||||
6. Report back with:
|
||||
- Device identity from console
|
||||
- Client ID from backend logs
|
||||
- Whether "🔔 Twilio Device INCOMING event" appears
|
||||
- Whether any accept logs appear
|
||||
|
||||
## Important Files
|
||||
- Backend: `/backend/src/voice/voice.controller.ts` (lines 205-210 show Client ID logging)
|
||||
- Frontend: `/frontend/composables/useSoftphone.ts` (Device initialization and incoming handler)
|
||||
173
SOFTPHONE_AI_ASSISTANT.md
Normal file
173
SOFTPHONE_AI_ASSISTANT.md
Normal file
@@ -0,0 +1,173 @@
|
||||
# Softphone AI Assistant - Complete Implementation
|
||||
|
||||
## 🎉 Features Implemented
|
||||
|
||||
### ✅ Real-time AI Call Assistant
|
||||
- **OpenAI Realtime API Integration** - Listens to live calls and provides suggestions
|
||||
- **Audio Streaming** - Twilio Media Streams fork audio to backend for AI processing
|
||||
- **Real-time Transcription** - Speech-to-text during calls
|
||||
- **Smart Suggestions** - AI analyzes conversation and advises the agent
|
||||
|
||||
## 🔧 Architecture
|
||||
|
||||
### Backend Flow
|
||||
```
|
||||
Inbound Call → TwiML (<Start><Stream> + <Dial>)
|
||||
→ Media Stream WebSocket → OpenAI Realtime API
|
||||
→ AI Processing → Socket.IO → Frontend
|
||||
```
|
||||
|
||||
### Key Components
|
||||
|
||||
1. **TwiML Structure** (`voice.controller.ts:226-234`)
|
||||
- `<Start><Stream>` - Forks audio for AI processing
|
||||
- `<Dial><Client>` - Connects call to agent's softphone
|
||||
|
||||
2. **OpenAI Integration** (`voice.service.ts:431-519`)
|
||||
- WebSocket connection to `wss://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview-2024-10-01`
|
||||
- Session config with custom instructions for agent assistance
|
||||
- Handles transcripts and generates suggestions
|
||||
|
||||
3. **AI Message Handler** (`voice.service.ts:609-707`)
|
||||
- Processes OpenAI events (transcripts, suggestions, audio)
|
||||
- Routes suggestions to frontend via Socket.IO
|
||||
- Saves transcripts to database
|
||||
|
||||
4. **Voice Gateway** (`voice.gateway.ts:272-289`)
|
||||
- `notifyAiTranscript()` - Real-time transcript chunks
|
||||
- `notifyAiSuggestion()` - AI suggestions to agent
|
||||
|
||||
### Frontend Components
|
||||
|
||||
1. **Softphone Dialog** (`SoftphoneDialog.vue:104-135`)
|
||||
- AI Assistant section with badge showing suggestion count
|
||||
- Color-coded suggestions (blue=response, green=action, purple=insight)
|
||||
- Animated highlight for newest suggestion
|
||||
|
||||
2. **Softphone Composable** (`useSoftphone.ts:515-535`)
|
||||
- Socket.IO event handlers for `ai:suggestion` and `ai:transcript`
|
||||
- Maintains history of last 10 suggestions
|
||||
- Maintains history of last 50 transcript items
|
||||
|
||||
## 📋 AI Prompt Configuration
|
||||
|
||||
The AI is instructed to:
|
||||
- **Listen, not talk** - It advises the agent, not the caller
|
||||
- **Provide concise suggestions** - 1-2 sentences max
|
||||
- **Use formatted output**:
|
||||
- `💡 Suggestion: [advice]`
|
||||
- `⚠️ Alert: [important notice]`
|
||||
- `📋 Action: [CRM action]`
|
||||
|
||||
## 🎨 UI Features
|
||||
|
||||
### Suggestion Types
|
||||
- **Response** (Blue) - Suggested replies or approaches
|
||||
- **Action** (Green) - Recommended CRM actions
|
||||
- **Insight** (Purple) - Important alerts or observations
|
||||
|
||||
### Visual Feedback
|
||||
- Badge showing number of suggestions
|
||||
- Newest suggestion pulses for attention
|
||||
- Auto-scrolling suggestion list
|
||||
- Timestamp on each suggestion
|
||||
|
||||
## 🔍 How to Monitor
|
||||
|
||||
### 1. Backend Logs
|
||||
```bash
|
||||
# Watch for AI events
|
||||
docker logs -f neo-backend-1 | grep -E "AI|OpenAI|transcript|suggestion"
|
||||
```
|
||||
|
||||
Key log markers:
|
||||
- `📝 Transcript chunk:` - Real-time speech detection
|
||||
- `✅ Final transcript:` - Complete transcript saved
|
||||
- `💡 AI Suggestion:` - AI-generated advice
|
||||
|
||||
### 2. Database
|
||||
```sql
|
||||
-- View call transcripts
|
||||
SELECT call_sid, ai_transcript, created_at
|
||||
FROM calls
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 5;
|
||||
```
|
||||
|
||||
### 3. Frontend Console
|
||||
- Open browser DevTools Console
|
||||
- Watch for: "AI suggestion:", "AI transcript:"
|
||||
|
||||
## 🚀 Testing
|
||||
|
||||
1. **Make a test call** to your Twilio number
|
||||
2. **Accept the call** in the softphone dialog
|
||||
3. **Talk during the call** - Say something like "I need to schedule a follow-up"
|
||||
4. **Watch the UI** - AI suggestions appear in real-time
|
||||
5. **Check logs** - See transcription and suggestion generation
|
||||
|
||||
## 📊 Current Status
|
||||
|
||||
✅ **Working**:
|
||||
- Inbound calls ring softphone
|
||||
- Media stream forks audio to backend
|
||||
- OpenAI processes audio (1300+ packets/call)
|
||||
- AI generates suggestions
|
||||
- Suggestions appear in frontend
|
||||
- Transcripts saved to database
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
### Required Environment Variables
|
||||
```env
|
||||
# OpenAI API Key (set in tenant integrations config)
|
||||
OPENAI_API_KEY=sk-...
|
||||
|
||||
# Optional overrides
|
||||
OPENAI_MODEL=gpt-4o-realtime-preview-2024-10-01
|
||||
OPENAI_VOICE=alloy
|
||||
```
|
||||
|
||||
### Tenant Configuration
|
||||
Set in Settings > Integrations:
|
||||
- OpenAI API Key
|
||||
- Model (optional)
|
||||
- Voice (optional)
|
||||
|
||||
## 🎯 Next Steps (Optional Enhancements)
|
||||
|
||||
1. **CRM Tool Execution** - Implement actual tool calls (search contacts, create tasks)
|
||||
2. **Audio Response** - Send OpenAI audio back to caller (two-way AI interaction)
|
||||
3. **Sentiment Analysis** - Track call sentiment in real-time
|
||||
4. **Call Summary** - Generate post-call summary automatically
|
||||
5. **Custom Prompts** - Allow agents to customize AI instructions per call type
|
||||
|
||||
## 🐛 Troubleshooting
|
||||
|
||||
### No suggestions appearing?
|
||||
1. Check OpenAI API key is configured
|
||||
2. Verify WebSocket connection logs show "OpenAI Realtime connected"
|
||||
3. Check frontend Socket.IO connection is established
|
||||
4. Verify user ID matches between backend and frontend
|
||||
|
||||
### Transcripts not saving?
|
||||
1. Check tenant database connection
|
||||
2. Verify `calls` table has `ai_transcript` column
|
||||
3. Check logs for "Failed to update transcript" errors
|
||||
|
||||
### OpenAI connection fails?
|
||||
1. Verify API key is valid
|
||||
2. Check model name is correct
|
||||
3. Review WebSocket close codes in logs
|
||||
|
||||
## 📝 Files Modified
|
||||
|
||||
**Backend:**
|
||||
- `/backend/src/voice/voice.service.ts` - OpenAI integration & AI message handling
|
||||
- `/backend/src/voice/voice.controller.ts` - TwiML generation with stream fork
|
||||
- `/backend/src/voice/voice.gateway.ts` - Socket.IO event emission
|
||||
- `/backend/src/main.ts` - Media stream WebSocket handler
|
||||
|
||||
**Frontend:**
|
||||
- `/frontend/components/SoftphoneDialog.vue` - AI suggestions UI
|
||||
- `/frontend/composables/useSoftphone.ts` - Socket.IO event handlers
|
||||
@@ -18,3 +18,6 @@ JWT_EXPIRES_IN="7d"
|
||||
# Application
|
||||
NODE_ENV="development"
|
||||
PORT="3000"
|
||||
|
||||
# Central Admin Subdomains (comma-separated list of subdomains that access the central database)
|
||||
CENTRAL_SUBDOMAINS="central,admin"
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
exports.up = function (knex) {
|
||||
return knex.schema.createTable('custom_migrations', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.uuid('tenantId').notNullable();
|
||||
table.string('name', 255).notNullable();
|
||||
table.text('description');
|
||||
table.enum('type', [
|
||||
'create_table',
|
||||
'add_column',
|
||||
'alter_column',
|
||||
'add_index',
|
||||
'drop_table',
|
||||
'custom',
|
||||
]).notNullable();
|
||||
table.text('sql').notNullable();
|
||||
table.enum('status', ['pending', 'executed', 'failed']).defaultTo('pending');
|
||||
table.timestamp('executedAt').nullable();
|
||||
table.text('error').nullable();
|
||||
table.timestamps(true, true);
|
||||
|
||||
table.index(['tenantId']);
|
||||
table.index(['status']);
|
||||
table.index(['created_at']);
|
||||
});
|
||||
};
|
||||
|
||||
exports.down = function (knex) {
|
||||
return knex.schema.dropTableIfExists('custom_migrations');
|
||||
};
|
||||
@@ -0,0 +1,103 @@
|
||||
exports.up = function (knex) {
|
||||
return knex.schema
|
||||
// Add orgWideDefault to object_definitions
|
||||
.alterTable('object_definitions', (table) => {
|
||||
table
|
||||
.enum('orgWideDefault', ['private', 'public_read', 'public_read_write'])
|
||||
.defaultTo('private')
|
||||
.notNullable();
|
||||
})
|
||||
// Create role_object_permissions table
|
||||
.createTable('role_object_permissions', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.uuid('roleId').notNullable();
|
||||
table.uuid('objectDefinitionId').notNullable();
|
||||
table.boolean('canCreate').defaultTo(false);
|
||||
table.boolean('canRead').defaultTo(false);
|
||||
table.boolean('canEdit').defaultTo(false);
|
||||
table.boolean('canDelete').defaultTo(false);
|
||||
table.boolean('canViewAll').defaultTo(false);
|
||||
table.boolean('canModifyAll').defaultTo(false);
|
||||
table.timestamps(true, true);
|
||||
|
||||
table
|
||||
.foreign('roleId')
|
||||
.references('id')
|
||||
.inTable('roles')
|
||||
.onDelete('CASCADE');
|
||||
table
|
||||
.foreign('objectDefinitionId')
|
||||
.references('id')
|
||||
.inTable('object_definitions')
|
||||
.onDelete('CASCADE');
|
||||
table.unique(['roleId', 'objectDefinitionId']);
|
||||
table.index(['roleId']);
|
||||
table.index(['objectDefinitionId']);
|
||||
})
|
||||
// Create role_field_permissions table
|
||||
.createTable('role_field_permissions', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.uuid('roleId').notNullable();
|
||||
table.uuid('fieldDefinitionId').notNullable();
|
||||
table.boolean('canRead').defaultTo(true);
|
||||
table.boolean('canEdit').defaultTo(true);
|
||||
table.timestamps(true, true);
|
||||
|
||||
table
|
||||
.foreign('roleId')
|
||||
.references('id')
|
||||
.inTable('roles')
|
||||
.onDelete('CASCADE');
|
||||
table
|
||||
.foreign('fieldDefinitionId')
|
||||
.references('id')
|
||||
.inTable('field_definitions')
|
||||
.onDelete('CASCADE');
|
||||
table.unique(['roleId', 'fieldDefinitionId']);
|
||||
table.index(['roleId']);
|
||||
table.index(['fieldDefinitionId']);
|
||||
})
|
||||
// Create record_shares table for sharing specific records
|
||||
.createTable('record_shares', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.uuid('objectDefinitionId').notNullable();
|
||||
table.uuid('recordId').notNullable();
|
||||
table.uuid('granteeUserId').notNullable();
|
||||
table.uuid('grantedByUserId').notNullable();
|
||||
table.json('accessLevel').notNullable(); // { canRead, canEdit, canDelete }
|
||||
table.timestamp('expiresAt').nullable();
|
||||
table.timestamp('revokedAt').nullable();
|
||||
table.timestamp('createdAt').defaultTo(knex.fn.now());
|
||||
table.timestamp('updatedAt').defaultTo(knex.fn.now());
|
||||
|
||||
table
|
||||
.foreign('objectDefinitionId')
|
||||
.references('id')
|
||||
.inTable('object_definitions')
|
||||
.onDelete('CASCADE');
|
||||
table
|
||||
.foreign('granteeUserId')
|
||||
.references('id')
|
||||
.inTable('users')
|
||||
.onDelete('CASCADE');
|
||||
table
|
||||
.foreign('grantedByUserId')
|
||||
.references('id')
|
||||
.inTable('users')
|
||||
.onDelete('CASCADE');
|
||||
table.index(['objectDefinitionId', 'recordId']);
|
||||
table.index(['granteeUserId']);
|
||||
table.index(['expiresAt']);
|
||||
table.index(['revokedAt']);
|
||||
});
|
||||
};
|
||||
|
||||
exports.down = function (knex) {
|
||||
return knex.schema
|
||||
.dropTableIfExists('record_shares')
|
||||
.dropTableIfExists('role_field_permissions')
|
||||
.dropTableIfExists('role_object_permissions')
|
||||
.alterTable('object_definitions', (table) => {
|
||||
table.dropColumn('orgWideDefault');
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,95 @@
|
||||
/**
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.up = async function(knex) {
|
||||
// Check if layout_type column already exists (in case of partial migration)
|
||||
const hasLayoutType = await knex.schema.hasColumn('page_layouts', 'layout_type');
|
||||
|
||||
// Check if the old index exists
|
||||
const [indexes] = await knex.raw(`SHOW INDEX FROM page_layouts WHERE Key_name = 'page_layouts_object_id_is_default_index'`);
|
||||
const hasOldIndex = indexes.length > 0;
|
||||
|
||||
// Check if foreign key exists
|
||||
const [fks] = await knex.raw(`
|
||||
SELECT CONSTRAINT_NAME FROM information_schema.TABLE_CONSTRAINTS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'page_layouts'
|
||||
AND CONSTRAINT_TYPE = 'FOREIGN KEY'
|
||||
AND CONSTRAINT_NAME = 'page_layouts_object_id_foreign'
|
||||
`);
|
||||
const hasForeignKey = fks.length > 0;
|
||||
|
||||
if (hasOldIndex) {
|
||||
// First, drop the foreign key constraint that depends on the index (if it exists)
|
||||
if (hasForeignKey) {
|
||||
await knex.schema.alterTable('page_layouts', (table) => {
|
||||
table.dropForeign(['object_id']);
|
||||
});
|
||||
}
|
||||
|
||||
// Now we can safely drop the old index
|
||||
await knex.schema.alterTable('page_layouts', (table) => {
|
||||
table.dropIndex(['object_id', 'is_default']);
|
||||
});
|
||||
}
|
||||
|
||||
// Add layout_type column if it doesn't exist
|
||||
if (!hasLayoutType) {
|
||||
await knex.schema.alterTable('page_layouts', (table) => {
|
||||
// Add layout_type column to distinguish between detail/edit layouts and list view layouts
|
||||
// Default to 'detail' for existing layouts
|
||||
table.enum('layout_type', ['detail', 'list']).notNullable().defaultTo('detail').after('name');
|
||||
});
|
||||
}
|
||||
|
||||
// Check if new index exists
|
||||
const [newIndexes] = await knex.raw(`SHOW INDEX FROM page_layouts WHERE Key_name = 'page_layouts_object_id_layout_type_is_default_index'`);
|
||||
const hasNewIndex = newIndexes.length > 0;
|
||||
|
||||
if (!hasNewIndex) {
|
||||
// Create new index including layout_type
|
||||
await knex.schema.alterTable('page_layouts', (table) => {
|
||||
table.index(['object_id', 'layout_type', 'is_default']);
|
||||
});
|
||||
}
|
||||
|
||||
// Re-check if foreign key exists (may have been dropped above or in previous attempt)
|
||||
const [fksAfter] = await knex.raw(`
|
||||
SELECT CONSTRAINT_NAME FROM information_schema.TABLE_CONSTRAINTS
|
||||
WHERE TABLE_SCHEMA = DATABASE()
|
||||
AND TABLE_NAME = 'page_layouts'
|
||||
AND CONSTRAINT_TYPE = 'FOREIGN KEY'
|
||||
AND CONSTRAINT_NAME = 'page_layouts_object_id_foreign'
|
||||
`);
|
||||
|
||||
if (fksAfter.length === 0) {
|
||||
// Re-add the foreign key constraint
|
||||
await knex.schema.alterTable('page_layouts', (table) => {
|
||||
table.foreign('object_id').references('id').inTable('object_definitions').onDelete('CASCADE');
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.down = async function(knex) {
|
||||
// Drop the foreign key first
|
||||
await knex.schema.alterTable('page_layouts', (table) => {
|
||||
table.dropForeign(['object_id']);
|
||||
});
|
||||
|
||||
// Drop the new index and column, restore old index
|
||||
await knex.schema.alterTable('page_layouts', (table) => {
|
||||
table.dropIndex(['object_id', 'layout_type', 'is_default']);
|
||||
table.dropColumn('layout_type');
|
||||
table.index(['object_id', 'is_default']);
|
||||
});
|
||||
|
||||
// Re-add the foreign key constraint
|
||||
await knex.schema.alterTable('page_layouts', (table) => {
|
||||
table.foreign('object_id').references('id').inTable('object_definitions').onDelete('CASCADE');
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,55 @@
|
||||
/**
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.up = async function (knex) {
|
||||
// Create calls table for tracking voice calls
|
||||
await knex.schema.createTable('calls', (table) => {
|
||||
table.string('id', 36).primary();
|
||||
table.string('call_sid', 100).unique().notNullable().comment('Twilio call SID');
|
||||
table.enum('direction', ['inbound', 'outbound']).notNullable();
|
||||
table.string('from_number', 20).notNullable();
|
||||
table.string('to_number', 20).notNullable();
|
||||
table.enum('status', [
|
||||
'queued',
|
||||
'ringing',
|
||||
'in-progress',
|
||||
'completed',
|
||||
'busy',
|
||||
'failed',
|
||||
'no-answer',
|
||||
'canceled'
|
||||
]).notNullable().defaultTo('queued');
|
||||
table.integer('duration_seconds').unsigned().nullable();
|
||||
table.string('recording_url', 500).nullable();
|
||||
table.text('ai_transcript').nullable().comment('Full transcript from OpenAI');
|
||||
table.text('ai_summary').nullable().comment('AI-generated summary');
|
||||
table.json('ai_insights').nullable().comment('Structured insights from AI');
|
||||
table.string('user_id', 36).notNullable().comment('User who handled the call');
|
||||
table.timestamp('started_at').nullable();
|
||||
table.timestamp('ended_at').nullable();
|
||||
table.timestamp('created_at').defaultTo(knex.fn.now());
|
||||
table.timestamp('updated_at').defaultTo(knex.fn.now());
|
||||
|
||||
// Indexes
|
||||
table.index('call_sid');
|
||||
table.index('user_id');
|
||||
table.index('status');
|
||||
table.index('direction');
|
||||
table.index(['created_at', 'user_id']);
|
||||
|
||||
// Foreign key to users table
|
||||
table.foreign('user_id').references('id').inTable('users').onDelete('CASCADE');
|
||||
});
|
||||
|
||||
console.log('✅ Created calls table');
|
||||
};
|
||||
|
||||
/**
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.down = async function (knex) {
|
||||
await knex.schema.dropTableIfExists('calls');
|
||||
console.log('✅ Dropped calls table');
|
||||
};
|
||||
@@ -0,0 +1,207 @@
|
||||
exports.up = async function (knex) {
|
||||
await knex.schema.createTable('contacts', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.string('firstName', 100).notNullable();
|
||||
table.string('lastName', 100).notNullable();
|
||||
table.uuid('accountId').notNullable();
|
||||
table.timestamps(true, true);
|
||||
|
||||
table
|
||||
.foreign('accountId')
|
||||
.references('id')
|
||||
.inTable('accounts')
|
||||
.onDelete('CASCADE');
|
||||
table.index(['accountId']);
|
||||
table.index(['lastName', 'firstName']);
|
||||
});
|
||||
|
||||
await knex.schema.createTable('contact_details', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.string('relatedObjectType', 100).notNullable();
|
||||
table.uuid('relatedObjectId').notNullable();
|
||||
table.string('detailType', 50).notNullable();
|
||||
table.string('label', 100);
|
||||
table.text('value').notNullable();
|
||||
table.boolean('isPrimary').defaultTo(false);
|
||||
table.timestamps(true, true);
|
||||
|
||||
table.index(['relatedObjectType', 'relatedObjectId']);
|
||||
table.index(['detailType']);
|
||||
});
|
||||
|
||||
const [contactObjectId] = await knex('object_definitions').insert({
|
||||
id: knex.raw('(UUID())'),
|
||||
apiName: 'Contact',
|
||||
label: 'Contact',
|
||||
pluralLabel: 'Contacts',
|
||||
description: 'Standard Contact object',
|
||||
isSystem: true,
|
||||
isCustom: false,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
const contactObjectDefId =
|
||||
contactObjectId ||
|
||||
(await knex('object_definitions').where('apiName', 'Contact').first()).id;
|
||||
|
||||
await knex('field_definitions').insert([
|
||||
{
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactObjectDefId,
|
||||
apiName: 'firstName',
|
||||
label: 'First Name',
|
||||
type: 'String',
|
||||
length: 100,
|
||||
isRequired: true,
|
||||
isSystem: true,
|
||||
isCustom: false,
|
||||
displayOrder: 1,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
},
|
||||
{
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactObjectDefId,
|
||||
apiName: 'lastName',
|
||||
label: 'Last Name',
|
||||
type: 'String',
|
||||
length: 100,
|
||||
isRequired: true,
|
||||
isSystem: true,
|
||||
isCustom: false,
|
||||
displayOrder: 2,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
},
|
||||
{
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactObjectDefId,
|
||||
apiName: 'accountId',
|
||||
label: 'Account',
|
||||
type: 'Reference',
|
||||
referenceObject: 'Account',
|
||||
isRequired: true,
|
||||
isSystem: true,
|
||||
isCustom: false,
|
||||
displayOrder: 3,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
},
|
||||
]);
|
||||
|
||||
const [contactDetailObjectId] = await knex('object_definitions').insert({
|
||||
id: knex.raw('(UUID())'),
|
||||
apiName: 'ContactDetail',
|
||||
label: 'Contact Detail',
|
||||
pluralLabel: 'Contact Details',
|
||||
description: 'Polymorphic contact detail object',
|
||||
isSystem: true,
|
||||
isCustom: false,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
const contactDetailObjectDefId =
|
||||
contactDetailObjectId ||
|
||||
(await knex('object_definitions').where('apiName', 'ContactDetail').first())
|
||||
.id;
|
||||
|
||||
const contactDetailRelationObjects = ['Account', 'Contact']
|
||||
|
||||
await knex('field_definitions').insert([
|
||||
{
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactDetailObjectDefId,
|
||||
apiName: 'relatedObjectType',
|
||||
label: 'Related Object Type',
|
||||
type: 'PICKLIST',
|
||||
length: 100,
|
||||
isRequired: true,
|
||||
isSystem: false,
|
||||
isCustom: false,
|
||||
displayOrder: 1,
|
||||
ui_metadata: JSON.stringify({
|
||||
options: contactDetailRelationObjects.map((value) => ({ label: value, value })),
|
||||
}),
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
},
|
||||
{
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactDetailObjectDefId,
|
||||
apiName: 'relatedObjectId',
|
||||
label: 'Related Object ID',
|
||||
type: 'LOOKUP',
|
||||
length: 36,
|
||||
isRequired: true,
|
||||
isSystem: false,
|
||||
isCustom: false,
|
||||
displayOrder: 2,
|
||||
ui_metadata: JSON.stringify({
|
||||
relationObjects: contactDetailRelationObjects,
|
||||
relationTypeField: 'relatedObjectType',
|
||||
relationDisplayField: 'name',
|
||||
}),
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
},
|
||||
{
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactDetailObjectDefId,
|
||||
apiName: 'detailType',
|
||||
label: 'Detail Type',
|
||||
type: 'String',
|
||||
length: 50,
|
||||
isRequired: true,
|
||||
isSystem: false,
|
||||
isCustom: false,
|
||||
displayOrder: 3,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
},
|
||||
{
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactDetailObjectDefId,
|
||||
apiName: 'label',
|
||||
label: 'Label',
|
||||
type: 'String',
|
||||
length: 100,
|
||||
isSystem: false,
|
||||
isCustom: false,
|
||||
displayOrder: 4,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
},
|
||||
{
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactDetailObjectDefId,
|
||||
apiName: 'value',
|
||||
label: 'Value',
|
||||
type: 'Text',
|
||||
isRequired: true,
|
||||
isSystem: false,
|
||||
isCustom: false,
|
||||
displayOrder: 5,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
},
|
||||
{
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactDetailObjectDefId,
|
||||
apiName: 'isPrimary',
|
||||
label: 'Primary',
|
||||
type: 'Boolean',
|
||||
isSystem: false,
|
||||
isCustom: false,
|
||||
displayOrder: 6,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
},
|
||||
]);
|
||||
};
|
||||
|
||||
exports.down = async function (knex) {
|
||||
await knex.schema.dropTableIfExists('contact_details');
|
||||
await knex.schema.dropTableIfExists('contacts');
|
||||
};
|
||||
@@ -0,0 +1,101 @@
|
||||
exports.up = async function (knex) {
|
||||
const contactDetailObject = await knex('object_definitions')
|
||||
.where({ apiName: 'ContactDetail' })
|
||||
.first();
|
||||
|
||||
if (!contactDetailObject) return;
|
||||
|
||||
const relationObjects = ['Account', 'Contact'];
|
||||
|
||||
await knex('field_definitions')
|
||||
.where({
|
||||
objectDefinitionId: contactDetailObject.id,
|
||||
apiName: 'relatedObjectType',
|
||||
})
|
||||
.update({
|
||||
type: 'PICKLIST',
|
||||
length: 100,
|
||||
isSystem: false,
|
||||
ui_metadata: JSON.stringify({
|
||||
options: relationObjects.map((value) => ({ label: value, value })),
|
||||
}),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
await knex('field_definitions')
|
||||
.where({
|
||||
objectDefinitionId: contactDetailObject.id,
|
||||
apiName: 'relatedObjectId',
|
||||
})
|
||||
.update({
|
||||
type: 'LOOKUP',
|
||||
length: 36,
|
||||
isSystem: false,
|
||||
ui_metadata: JSON.stringify({
|
||||
relationObjects,
|
||||
relationTypeField: 'relatedObjectType',
|
||||
relationDisplayField: 'name',
|
||||
}),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
await knex('field_definitions')
|
||||
.whereIn('apiName', [
|
||||
'detailType',
|
||||
'label',
|
||||
'value',
|
||||
'isPrimary',
|
||||
])
|
||||
.andWhere({ objectDefinitionId: contactDetailObject.id })
|
||||
.update({
|
||||
isSystem: false,
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
};
|
||||
|
||||
exports.down = async function (knex) {
|
||||
const contactDetailObject = await knex('object_definitions')
|
||||
.where({ apiName: 'ContactDetail' })
|
||||
.first();
|
||||
|
||||
if (!contactDetailObject) return;
|
||||
|
||||
await knex('field_definitions')
|
||||
.where({
|
||||
objectDefinitionId: contactDetailObject.id,
|
||||
apiName: 'relatedObjectType',
|
||||
})
|
||||
.update({
|
||||
type: 'String',
|
||||
length: 100,
|
||||
isSystem: true,
|
||||
ui_metadata: null,
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
await knex('field_definitions')
|
||||
.where({
|
||||
objectDefinitionId: contactDetailObject.id,
|
||||
apiName: 'relatedObjectId',
|
||||
})
|
||||
.update({
|
||||
type: 'String',
|
||||
length: 36,
|
||||
isSystem: true,
|
||||
ui_metadata: null,
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
await knex('field_definitions')
|
||||
.whereIn('apiName', [
|
||||
'detailType',
|
||||
'label',
|
||||
'value',
|
||||
'isPrimary',
|
||||
])
|
||||
.andWhere({ objectDefinitionId: contactDetailObject.id })
|
||||
.update({
|
||||
isSystem: true,
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,45 @@
|
||||
exports.up = async function (knex) {
|
||||
const contactDetailObject = await knex('object_definitions')
|
||||
.where({ apiName: 'ContactDetail' })
|
||||
.first();
|
||||
|
||||
if (!contactDetailObject) return;
|
||||
|
||||
await knex('field_definitions')
|
||||
.where({ objectDefinitionId: contactDetailObject.id })
|
||||
.whereIn('apiName', [
|
||||
'relatedObjectType',
|
||||
'relatedObjectId',
|
||||
'detailType',
|
||||
'label',
|
||||
'value',
|
||||
'isPrimary',
|
||||
])
|
||||
.update({
|
||||
isSystem: false,
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
};
|
||||
|
||||
exports.down = async function (knex) {
|
||||
const contactDetailObject = await knex('object_definitions')
|
||||
.where({ apiName: 'ContactDetail' })
|
||||
.first();
|
||||
|
||||
if (!contactDetailObject) return;
|
||||
|
||||
await knex('field_definitions')
|
||||
.where({ objectDefinitionId: contactDetailObject.id })
|
||||
.whereIn('apiName', [
|
||||
'relatedObjectType',
|
||||
'relatedObjectId',
|
||||
'detailType',
|
||||
'label',
|
||||
'value',
|
||||
'isPrimary',
|
||||
])
|
||||
.update({
|
||||
isSystem: true,
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,62 @@
|
||||
exports.up = async function (knex) {
|
||||
// Add ownerId column to contacts
|
||||
await knex.schema.alterTable('contacts', (table) => {
|
||||
table.uuid('ownerId');
|
||||
table
|
||||
.foreign('ownerId')
|
||||
.references('id')
|
||||
.inTable('users')
|
||||
.onDelete('SET NULL');
|
||||
table.index(['ownerId']);
|
||||
});
|
||||
|
||||
// Add ownerId field definition metadata for Contact object
|
||||
const contactObject = await knex('object_definitions')
|
||||
.where('apiName', 'Contact')
|
||||
.first();
|
||||
|
||||
if (contactObject) {
|
||||
const existingField = await knex('field_definitions')
|
||||
.where({
|
||||
objectDefinitionId: contactObject.id,
|
||||
apiName: 'ownerId',
|
||||
})
|
||||
.first();
|
||||
|
||||
if (!existingField) {
|
||||
await knex('field_definitions').insert({
|
||||
id: knex.raw('(UUID())'),
|
||||
objectDefinitionId: contactObject.id,
|
||||
apiName: 'ownerId',
|
||||
label: 'Owner',
|
||||
type: 'Reference',
|
||||
referenceObject: 'User',
|
||||
isSystem: true,
|
||||
isCustom: false,
|
||||
displayOrder: 4,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
exports.down = async function (knex) {
|
||||
const contactObject = await knex('object_definitions')
|
||||
.where('apiName', 'Contact')
|
||||
.first();
|
||||
|
||||
if (contactObject) {
|
||||
await knex('field_definitions')
|
||||
.where({
|
||||
objectDefinitionId: contactObject.id,
|
||||
apiName: 'ownerId',
|
||||
})
|
||||
.delete();
|
||||
}
|
||||
|
||||
await knex.schema.alterTable('contacts', (table) => {
|
||||
table.dropForeign(['ownerId']);
|
||||
table.dropColumn('ownerId');
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,55 @@
|
||||
/**
|
||||
* Creates the saved_list_views table.
|
||||
* Each row stores a named, reusable search/filter configuration for a specific
|
||||
* CRM object type. Views can be private to the owning user or shared with the
|
||||
* whole tenant.
|
||||
*
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.up = function (knex) {
|
||||
return knex.schema.createTable('saved_list_views', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
|
||||
// Human-readable name given by the user (or AI-suggested)
|
||||
table.string('name').notNullable();
|
||||
|
||||
// The object this view belongs to (e.g. "Dog", "Contact")
|
||||
table.string('object_api_name').notNullable();
|
||||
|
||||
// The user who created/owns this view
|
||||
table.uuid('user_id').notNullable();
|
||||
|
||||
// When true the view is visible to all users in the tenant
|
||||
table.boolean('is_shared').notNullable().defaultTo(false);
|
||||
|
||||
// Strategy is always "query" for saved views (keyword views are not saved)
|
||||
table.string('strategy').notNullable().defaultTo('query');
|
||||
|
||||
// Resolved filters as JSON array of AiSearchFilter objects
|
||||
table.json('filters').notNullable();
|
||||
|
||||
// Optional sort: { field: string, direction: "asc" | "desc" }
|
||||
table.json('sort').nullable();
|
||||
|
||||
// AI-generated plain-language explanation of what this view shows
|
||||
table.text('description').nullable();
|
||||
|
||||
table.timestamps(true, true);
|
||||
|
||||
// Foreign key to users
|
||||
table.foreign('user_id').references('id').inTable('users').onDelete('CASCADE');
|
||||
|
||||
// Primary lookup: all views for an object visible to a user
|
||||
table.index(['object_api_name', 'user_id']);
|
||||
table.index(['object_api_name', 'is_shared']);
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.down = function (knex) {
|
||||
return knex.schema.dropTableIfExists('saved_list_views');
|
||||
};
|
||||
@@ -0,0 +1,35 @@
|
||||
/**
|
||||
* Inserts a system object_definition row for SavedListView.
|
||||
* This allows saved_list_views records to be shared via record_shares
|
||||
* (which requires a valid objectDefinitionId FK).
|
||||
*
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.up = async function (knex) {
|
||||
// Only insert if it doesn't already exist (idempotent)
|
||||
const existing = await knex('object_definitions')
|
||||
.where({ apiName: 'SavedListView' })
|
||||
.first();
|
||||
|
||||
if (!existing) {
|
||||
await knex('object_definitions').insert({
|
||||
apiName: 'SavedListView',
|
||||
label: 'Saved List View',
|
||||
pluralLabel: 'Saved List Views',
|
||||
description: 'System object for sharing saved list views via record_shares',
|
||||
isSystem: true,
|
||||
isCustom: false,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.down = async function (knex) {
|
||||
await knex('object_definitions')
|
||||
.where({ apiName: 'SavedListView' })
|
||||
.delete();
|
||||
};
|
||||
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Add 'alias' and virtual 'name' column to users table.
|
||||
*
|
||||
* - alias: a user-editable display name / nickname
|
||||
* - name: a generated column that returns COALESCE(alias, CONCAT(firstName, ' ', lastName), email)
|
||||
* so that lookup fields referencing User.name always resolve.
|
||||
*/
|
||||
exports.up = function (knex) {
|
||||
return knex.schema.alterTable('users', (table) => {
|
||||
table.string('alias', 255).nullable().after('lastName');
|
||||
table.string('name', 512).nullable().after('alias');
|
||||
}).then(() => {
|
||||
// Backfill existing rows: name = alias, or firstName + lastName, or email
|
||||
return knex.raw(`
|
||||
UPDATE users
|
||||
SET name = COALESCE(
|
||||
NULLIF(alias, ''),
|
||||
NULLIF(TRIM(CONCAT(COALESCE(firstName, ''), ' ', COALESCE(lastName, ''))), ''),
|
||||
email
|
||||
)
|
||||
`);
|
||||
});
|
||||
};
|
||||
|
||||
exports.down = function (knex) {
|
||||
return knex.schema.alterTable('users', (table) => {
|
||||
table.dropColumn('name');
|
||||
table.dropColumn('alias');
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,93 @@
|
||||
/**
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.up = async function (knex) {
|
||||
await knex.schema.createTable('comments', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.string('parent_object_api_name').notNullable();
|
||||
table.uuid('parent_record_id').notNullable();
|
||||
table.uuid('author_user_id').notNullable();
|
||||
table.text('content').notNullable();
|
||||
table.timestamps(true, true);
|
||||
|
||||
table.foreign('author_user_id').references('id').inTable('users').onDelete('CASCADE');
|
||||
table.index(['parent_object_api_name', 'parent_record_id'], 'comments_parent_idx');
|
||||
table.index(['author_user_id'], 'comments_author_idx');
|
||||
});
|
||||
|
||||
await knex.schema.createTable('semantic_documents', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.string('entity_type').notNullable();
|
||||
table.uuid('entity_id').notNullable();
|
||||
table.string('title').nullable();
|
||||
table.text('narrative').nullable();
|
||||
table.json('metadata').nullable();
|
||||
table.json('source_summary').nullable();
|
||||
table.timestamps(true, true);
|
||||
|
||||
table.unique(['entity_type', 'entity_id'], {
|
||||
indexName: 'semantic_documents_entity_unique',
|
||||
});
|
||||
table.index(['entity_type'], 'semantic_documents_type_idx');
|
||||
});
|
||||
|
||||
await knex.schema.createTable('semantic_chunks', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.uuid('semantic_document_id').notNullable();
|
||||
table.integer('chunk_index').notNullable();
|
||||
table.string('source_kind').notNullable().defaultTo('base_record');
|
||||
table.uuid('source_ref_id').nullable();
|
||||
table.text('text').notNullable();
|
||||
table.json('metadata').nullable();
|
||||
table.timestamps(true, true);
|
||||
|
||||
table.foreign('semantic_document_id').references('id').inTable('semantic_documents').onDelete('CASCADE');
|
||||
table.unique(['semantic_document_id', 'chunk_index'], {
|
||||
indexName: 'semantic_chunks_doc_index_unique',
|
||||
});
|
||||
table.index(['semantic_document_id'], 'semantic_chunks_document_idx');
|
||||
table.index(['source_kind'], 'semantic_chunks_source_kind_idx');
|
||||
});
|
||||
|
||||
await knex.schema.createTable('semantic_links', (table) => {
|
||||
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
|
||||
table.string('source_entity_type', 100).notNullable();
|
||||
table.uuid('source_entity_id').notNullable();
|
||||
table.string('target_entity_type', 100).notNullable();
|
||||
table.uuid('target_entity_id').notNullable();
|
||||
table.string('link_type', 100).notNullable().defaultTo('related_to');
|
||||
table.string('status').notNullable().defaultTo('suggested');
|
||||
table.string('origin').notNullable().defaultTo('semantic');
|
||||
table.decimal('confidence', 5, 4).notNullable().defaultTo(0);
|
||||
table.text('reason').nullable();
|
||||
table.json('evidence').nullable();
|
||||
table.uuid('suggested_by_user_id').nullable();
|
||||
table.uuid('reviewed_by_user_id').nullable();
|
||||
table.timestamp('reviewed_at').nullable();
|
||||
table.timestamps(true, true);
|
||||
|
||||
table.foreign('suggested_by_user_id').references('id').inTable('users').onDelete('SET NULL');
|
||||
table.foreign('reviewed_by_user_id').references('id').inTable('users').onDelete('SET NULL');
|
||||
|
||||
table.unique(
|
||||
['source_entity_type', 'source_entity_id', 'target_entity_type', 'target_entity_id', 'link_type'],
|
||||
{ indexName: 'semantic_links_unique_pair_type' },
|
||||
);
|
||||
|
||||
table.index(['source_entity_type', 'source_entity_id'], 'semantic_links_source_idx');
|
||||
table.index(['target_entity_type', 'target_entity_id'], 'semantic_links_target_idx');
|
||||
table.index(['status'], 'semantic_links_status_idx');
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* @param { import("knex").Knex } knex
|
||||
* @returns { Promise<void> }
|
||||
*/
|
||||
exports.down = async function (knex) {
|
||||
await knex.schema.dropTableIfExists('semantic_links');
|
||||
await knex.schema.dropTableIfExists('semantic_chunks');
|
||||
await knex.schema.dropTableIfExists('semantic_documents');
|
||||
await knex.schema.dropTableIfExists('comments');
|
||||
};
|
||||
1290
backend/package-lock.json
generated
1290
backend/package-lock.json
generated
File diff suppressed because it is too large
Load Diff
@@ -26,6 +26,11 @@
|
||||
"migrate:all-tenants": "ts-node -r tsconfig-paths/register scripts/migrate-all-tenants.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@casl/ability": "^6.7.5",
|
||||
"@fastify/websocket": "^10.0.1",
|
||||
"@langchain/core": "^1.1.15",
|
||||
"@langchain/langgraph": "^1.0.15",
|
||||
"@langchain/openai": "^1.2.1",
|
||||
"@nestjs/bullmq": "^10.1.0",
|
||||
"@nestjs/common": "^10.3.0",
|
||||
"@nestjs/config": "^3.1.1",
|
||||
@@ -33,19 +38,28 @@
|
||||
"@nestjs/jwt": "^10.2.0",
|
||||
"@nestjs/passport": "^10.0.3",
|
||||
"@nestjs/platform-fastify": "^10.3.0",
|
||||
"@nestjs/platform-socket.io": "^10.4.20",
|
||||
"@nestjs/serve-static": "^4.0.2",
|
||||
"@nestjs/websockets": "^10.4.20",
|
||||
"@prisma/client": "^5.8.0",
|
||||
"bcrypt": "^5.1.1",
|
||||
"bullmq": "^5.1.0",
|
||||
"class-transformer": "^0.5.1",
|
||||
"class-validator": "^0.14.1",
|
||||
"deepagents": "^1.5.0",
|
||||
"ioredis": "^5.3.2",
|
||||
"knex": "^3.1.0",
|
||||
"langchain": "^1.2.10",
|
||||
"mysql2": "^3.15.3",
|
||||
"objection": "^3.1.5",
|
||||
"openai": "^6.15.0",
|
||||
"passport": "^0.7.0",
|
||||
"passport-jwt": "^4.0.1",
|
||||
"reflect-metadata": "^0.2.1",
|
||||
"rxjs": "^7.8.1"
|
||||
"rxjs": "^7.8.1",
|
||||
"socket.io": "^4.8.3",
|
||||
"twilio": "^5.11.1",
|
||||
"ws": "^8.18.3"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@nestjs/cli": "^10.3.0",
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
-- AlterTable
|
||||
ALTER TABLE `tenants` ADD COLUMN `integrationsConfig` JSON NULL;
|
||||
@@ -32,6 +32,7 @@ model Tenant {
|
||||
dbName String // Database name
|
||||
dbUsername String // Database username
|
||||
dbPassword String // Encrypted database password
|
||||
integrationsConfig Json? // Encrypted JSON config for external services (Twilio, OpenAI, etc.)
|
||||
status String @default("active") // active, suspended, deleted
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
@@ -20,6 +20,8 @@ model User {
|
||||
password String
|
||||
firstName String?
|
||||
lastName String?
|
||||
alias String?
|
||||
name String?
|
||||
isActive Boolean @default(true)
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
@@ -125,6 +127,7 @@ model FieldDefinition {
|
||||
isSystem Boolean @default(false)
|
||||
isCustom Boolean @default(true)
|
||||
displayOrder Int @default(0)
|
||||
uiMetadata Json? @map("ui_metadata")
|
||||
createdAt DateTime @default(now()) @map("created_at")
|
||||
updatedAt DateTime @updatedAt @map("updated_at")
|
||||
|
||||
@@ -145,11 +148,41 @@ model Account {
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
owner User @relation(fields: [ownerId], references: [id])
|
||||
contacts Contact[]
|
||||
|
||||
@@index([ownerId])
|
||||
@@map("accounts")
|
||||
}
|
||||
|
||||
model Contact {
|
||||
id String @id @default(uuid())
|
||||
firstName String
|
||||
lastName String
|
||||
accountId String
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
account Account @relation(fields: [accountId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@index([accountId])
|
||||
@@map("contacts")
|
||||
}
|
||||
|
||||
model ContactDetail {
|
||||
id String @id @default(uuid())
|
||||
relatedObjectType String
|
||||
relatedObjectId String
|
||||
detailType String
|
||||
label String?
|
||||
value String
|
||||
isPrimary Boolean @default(false)
|
||||
createdAt DateTime @default(now())
|
||||
updatedAt DateTime @updatedAt
|
||||
|
||||
@@index([relatedObjectType, relatedObjectId])
|
||||
@@map("contact_details")
|
||||
}
|
||||
|
||||
// Application Builder
|
||||
model App {
|
||||
id String @id @default(uuid())
|
||||
|
||||
@@ -1,8 +1,53 @@
|
||||
# Tenant Migration Scripts
|
||||
# Tenant Migration & Admin Scripts
|
||||
|
||||
This directory contains scripts for managing database migrations across all tenants in the multi-tenant platform.
|
||||
This directory contains scripts for managing database migrations across all tenants and creating admin users in the multi-tenant platform.
|
||||
|
||||
## Available Scripts
|
||||
## Admin User Management
|
||||
|
||||
### Create Central Admin User
|
||||
|
||||
```bash
|
||||
npm run create-central-admin
|
||||
```
|
||||
|
||||
Creates an administrator user in the **central database**. Central admins can:
|
||||
- Manage tenants (create, update, delete)
|
||||
- Access platform-wide administration features
|
||||
- View all tenant information
|
||||
- Manage tenant provisioning
|
||||
|
||||
**Interactive Mode:**
|
||||
```bash
|
||||
npm run create-central-admin
|
||||
# You will be prompted for:
|
||||
# - Email
|
||||
# - Password
|
||||
# - First Name (optional)
|
||||
# - Last Name (optional)
|
||||
# - Role (admin or superadmin)
|
||||
```
|
||||
|
||||
**Non-Interactive Mode (using environment variables):**
|
||||
```bash
|
||||
EMAIL=admin@example.com PASSWORD=securepass123 FIRST_NAME=John LAST_NAME=Doe ROLE=superadmin npm run create-central-admin
|
||||
```
|
||||
|
||||
**Logging In as Central Admin:**
|
||||
1. Access the application using a central subdomain (e.g., `central.yourdomain.com` or `admin.yourdomain.com`)
|
||||
2. Enter your central admin credentials
|
||||
3. You'll be authenticated against the central database (not a tenant database)
|
||||
|
||||
**Note:** The system automatically detects if you're logging in from a central subdomain based on the `CENTRAL_SUBDOMAINS` environment variable (defaults to `central,admin`). No special UI or configuration is needed on the frontend.
|
||||
|
||||
### Create Tenant User
|
||||
|
||||
For creating users within a specific tenant database, use:
|
||||
```bash
|
||||
npm run create-tenant-user <tenant-slug>
|
||||
# (Note: This script may need to be created or already exists)
|
||||
```
|
||||
|
||||
## Migration Scripts
|
||||
|
||||
### 1. Create a New Migration
|
||||
|
||||
|
||||
@@ -43,8 +43,9 @@ function decryptPassword(encryptedPassword: string): string {
|
||||
function createTenantKnexConnection(tenant: any): Knex {
|
||||
const decryptedPassword = decryptPassword(tenant.dbPassword);
|
||||
|
||||
// Replace 'db' hostname with 'localhost' when running outside Docker
|
||||
const dbHost = tenant.dbHost === 'db' ? 'localhost' : tenant.dbHost;
|
||||
// Use Docker hostname 'db' when running inside container
|
||||
// The dbHost will be 'db' for Docker connections or 'localhost' for local development
|
||||
const dbHost = tenant.dbHost;
|
||||
|
||||
return knex({
|
||||
client: 'mysql2',
|
||||
@@ -82,7 +83,7 @@ async function migrateTenant(tenant: any): Promise<void> {
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`❌ ${tenant.name}: Migration failed:`, error.message);
|
||||
console.error(`❌ ${tenant.name}: Migration failed:`, error);
|
||||
throw error;
|
||||
} finally {
|
||||
await tenantKnex.destroy();
|
||||
|
||||
181
backend/scripts/seed-default-roles.ts
Normal file
181
backend/scripts/seed-default-roles.ts
Normal file
@@ -0,0 +1,181 @@
|
||||
import { Knex } from 'knex';
|
||||
import * as knexLib from 'knex';
|
||||
|
||||
/**
|
||||
* Create a Knex connection for tenant database
|
||||
*/
|
||||
function createKnexConnection(database: string): Knex {
|
||||
return knexLib.default({
|
||||
client: 'mysql2',
|
||||
connection: {
|
||||
host: process.env.DB_HOST || 'db',
|
||||
port: parseInt(process.env.DB_PORT || '3306'),
|
||||
user: 'root',
|
||||
password: 'asjdnfqTash37faggT',
|
||||
database: database,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
interface RoleWithPermissions {
|
||||
name: string;
|
||||
description: string;
|
||||
objectPermissions: {
|
||||
[objectApiName: string]: {
|
||||
canCreate: boolean;
|
||||
canRead: boolean;
|
||||
canEdit: boolean;
|
||||
canDelete: boolean;
|
||||
canViewAll: boolean;
|
||||
canModifyAll: boolean;
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
const DEFAULT_ROLES: RoleWithPermissions[] = [
|
||||
{
|
||||
name: 'System Administrator',
|
||||
description: 'Full access to all objects and records. Can view and modify all data.',
|
||||
objectPermissions: {
|
||||
'*': {
|
||||
canCreate: true,
|
||||
canRead: true,
|
||||
canEdit: true,
|
||||
canDelete: true,
|
||||
canViewAll: true,
|
||||
canModifyAll: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'Standard User',
|
||||
description: 'Can create, read, edit, and delete own records. Respects OWD settings.',
|
||||
objectPermissions: {
|
||||
'*': {
|
||||
canCreate: true,
|
||||
canRead: true,
|
||||
canEdit: true,
|
||||
canDelete: true,
|
||||
canViewAll: false,
|
||||
canModifyAll: false,
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: 'Read Only',
|
||||
description: 'Can only read records based on OWD settings. No create, edit, or delete.',
|
||||
objectPermissions: {
|
||||
'*': {
|
||||
canCreate: false,
|
||||
canRead: true,
|
||||
canEdit: false,
|
||||
canDelete: false,
|
||||
canViewAll: false,
|
||||
canModifyAll: false,
|
||||
},
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
async function seedRolesForTenant(knex: Knex, tenantName: string) {
|
||||
console.log(`\n🌱 Seeding roles for tenant: ${tenantName}`);
|
||||
|
||||
// Get all object definitions
|
||||
const objectDefinitions = await knex('object_definitions').select('id', 'apiName');
|
||||
|
||||
for (const roleData of DEFAULT_ROLES) {
|
||||
// Check if role already exists
|
||||
const existingRole = await knex('roles')
|
||||
.where({ name: roleData.name })
|
||||
.first();
|
||||
|
||||
let roleId: string;
|
||||
|
||||
if (existingRole) {
|
||||
console.log(` ℹ️ Role "${roleData.name}" already exists, skipping...`);
|
||||
roleId = existingRole.id;
|
||||
} else {
|
||||
// Create role
|
||||
await knex('roles').insert({
|
||||
name: roleData.name,
|
||||
guardName: 'api',
|
||||
description: roleData.description,
|
||||
});
|
||||
|
||||
// Get the inserted role
|
||||
const newRole = await knex('roles')
|
||||
.where({ name: roleData.name })
|
||||
.first();
|
||||
|
||||
roleId = newRole.id;
|
||||
console.log(` ✅ Created role: ${roleData.name}`);
|
||||
}
|
||||
|
||||
// Create object permissions for all objects
|
||||
const wildcardPermissions = roleData.objectPermissions['*'];
|
||||
|
||||
for (const objectDef of objectDefinitions) {
|
||||
// Check if permission already exists
|
||||
const existingPermission = await knex('role_object_permissions')
|
||||
.where({
|
||||
roleId: roleId,
|
||||
objectDefinitionId: objectDef.id,
|
||||
})
|
||||
.first();
|
||||
|
||||
if (!existingPermission) {
|
||||
await knex('role_object_permissions').insert({
|
||||
roleId: roleId,
|
||||
objectDefinitionId: objectDef.id,
|
||||
canCreate: wildcardPermissions.canCreate,
|
||||
canRead: wildcardPermissions.canRead,
|
||||
canEdit: wildcardPermissions.canEdit,
|
||||
canDelete: wildcardPermissions.canDelete,
|
||||
canViewAll: wildcardPermissions.canViewAll,
|
||||
canModifyAll: wildcardPermissions.canModifyAll,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
console.log(` 📋 Set permissions for ${objectDefinitions.length} objects`);
|
||||
}
|
||||
}
|
||||
|
||||
async function seedAllTenants() {
|
||||
console.log('🚀 Starting role seeding for all tenants...\n');
|
||||
|
||||
// For now, seed the main tenant database
|
||||
const databases = ['tenant_tenant1'];
|
||||
|
||||
let successCount = 0;
|
||||
let errorCount = 0;
|
||||
|
||||
for (const database of databases) {
|
||||
try {
|
||||
const knex = createKnexConnection(database);
|
||||
await seedRolesForTenant(knex, database);
|
||||
await knex.destroy();
|
||||
successCount++;
|
||||
} catch (error) {
|
||||
console.error(`❌ ${database}: Seeding failed:`, error.message);
|
||||
errorCount++;
|
||||
}
|
||||
}
|
||||
|
||||
console.log('\n============================================================');
|
||||
console.log('📊 Seeding Summary');
|
||||
console.log('============================================================');
|
||||
console.log(`✅ Successful: ${successCount}`);
|
||||
console.log(`❌ Failed: ${errorCount}`);
|
||||
|
||||
if (errorCount === 0) {
|
||||
console.log('\n🎉 All tenant roles seeded successfully!');
|
||||
}
|
||||
}
|
||||
|
||||
seedAllTenants()
|
||||
.then(() => process.exit(0))
|
||||
.catch((error) => {
|
||||
console.error('Unhandled error:', error);
|
||||
process.exit(1);
|
||||
});
|
||||
49
backend/src/ai-assistant/ai-assistant.controller.ts
Normal file
49
backend/src/ai-assistant/ai-assistant.controller.ts
Normal file
@@ -0,0 +1,49 @@
|
||||
import { Body, Controller, Post, UseGuards } from '@nestjs/common';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { CurrentUser } from '../auth/current-user.decorator';
|
||||
import { TenantId } from '../tenant/tenant.decorator';
|
||||
import { AiAssistantService } from './ai-assistant.service';
|
||||
import { AiChatRequestDto } from './dto/ai-chat.dto';
|
||||
import { AiSearchRequestDto } from './dto/ai-search.dto';
|
||||
|
||||
@Controller('ai')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
export class AiAssistantController {
|
||||
constructor(private readonly aiAssistantService: AiAssistantService) {}
|
||||
|
||||
@Post('chat')
|
||||
async chat(
|
||||
@TenantId() tenantId: string,
|
||||
@CurrentUser() user: any,
|
||||
@Body() payload: AiChatRequestDto,
|
||||
) {
|
||||
return this.aiAssistantService.handleChat(
|
||||
tenantId,
|
||||
user.userId,
|
||||
payload.message,
|
||||
payload.history,
|
||||
payload.context,
|
||||
);
|
||||
}
|
||||
|
||||
@Post('search')
|
||||
async search(
|
||||
@TenantId() tenantId: string,
|
||||
@CurrentUser() user: any,
|
||||
@Body() payload: AiSearchRequestDto,
|
||||
) {
|
||||
return this.aiAssistantService.searchRecords(
|
||||
tenantId,
|
||||
user.userId,
|
||||
payload,
|
||||
);
|
||||
}
|
||||
|
||||
@Post('suggest-view-name')
|
||||
async suggestViewName(
|
||||
@TenantId() tenantId: string,
|
||||
@Body() payload: { objectLabel: string; filters: any[]; explanation?: string },
|
||||
) {
|
||||
return this.aiAssistantService.suggestViewName(tenantId, payload);
|
||||
}
|
||||
}
|
||||
14
backend/src/ai-assistant/ai-assistant.module.ts
Normal file
14
backend/src/ai-assistant/ai-assistant.module.ts
Normal file
@@ -0,0 +1,14 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { AiAssistantController } from './ai-assistant.controller';
|
||||
import { AiAssistantService } from './ai-assistant.service';
|
||||
import { ObjectModule } from '../object/object.module';
|
||||
import { PageLayoutModule } from '../page-layout/page-layout.module';
|
||||
import { TenantModule } from '../tenant/tenant.module';
|
||||
import { MeilisearchModule } from '../search/meilisearch.module';
|
||||
|
||||
@Module({
|
||||
imports: [ObjectModule, PageLayoutModule, TenantModule, MeilisearchModule],
|
||||
controllers: [AiAssistantController],
|
||||
providers: [AiAssistantService],
|
||||
})
|
||||
export class AiAssistantModule {}
|
||||
3108
backend/src/ai-assistant/ai-assistant.service.ts
Normal file
3108
backend/src/ai-assistant/ai-assistant.service.ts
Normal file
File diff suppressed because it is too large
Load Diff
111
backend/src/ai-assistant/ai-assistant.types.ts
Normal file
111
backend/src/ai-assistant/ai-assistant.types.ts
Normal file
@@ -0,0 +1,111 @@
|
||||
export interface AiChatMessage {
|
||||
role: 'user' | 'assistant';
|
||||
text: string;
|
||||
}
|
||||
|
||||
export interface AiChatContext {
|
||||
objectApiName?: string;
|
||||
view?: string;
|
||||
recordId?: string;
|
||||
route?: string;
|
||||
}
|
||||
|
||||
export interface AiAssistantReply {
|
||||
reply: string;
|
||||
action?: 'create_record' | 'collect_fields' | 'clarify' | 'plan_complete' | 'plan_pending';
|
||||
missingFields?: string[];
|
||||
record?: any;
|
||||
records?: any[]; // Multiple records when plan execution completes
|
||||
plan?: RecordCreationPlan;
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// Entity Discovery Types
|
||||
// ============================================
|
||||
|
||||
export interface EntityFieldInfo {
|
||||
apiName: string;
|
||||
label: string;
|
||||
type: string;
|
||||
isRequired: boolean;
|
||||
isSystem: boolean;
|
||||
referenceObject?: string; // For LOOKUP fields, the target entity
|
||||
description?: string;
|
||||
}
|
||||
|
||||
export interface EntityRelationship {
|
||||
fieldApiName: string;
|
||||
fieldLabel: string;
|
||||
targetEntity: string;
|
||||
relationshipType: 'lookup' | 'master-detail' | 'polymorphic';
|
||||
}
|
||||
|
||||
export interface EntityInfo {
|
||||
apiName: string;
|
||||
label: string;
|
||||
pluralLabel?: string;
|
||||
description?: string;
|
||||
fields: EntityFieldInfo[];
|
||||
requiredFields: string[]; // Field apiNames that are required
|
||||
relationships: EntityRelationship[];
|
||||
}
|
||||
|
||||
export interface SystemEntities {
|
||||
entities: EntityInfo[];
|
||||
entityByApiName: Record<string, EntityInfo>; // Changed from Map for state serialization
|
||||
loadedAt: number;
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// Planning Types
|
||||
// ============================================
|
||||
|
||||
export interface PlannedRecord {
|
||||
id: string; // Temporary ID for planning (e.g., "temp_account_1")
|
||||
entityApiName: string;
|
||||
entityLabel: string;
|
||||
fields: Record<string, any>;
|
||||
resolvedFields?: Record<string, any>; // Fields after dependency resolution
|
||||
missingRequiredFields: string[];
|
||||
dependsOn: string[]; // IDs of other planned records this depends on
|
||||
status: 'pending' | 'ready' | 'created' | 'failed';
|
||||
createdRecordId?: string; // Actual ID after creation
|
||||
wasExisting?: boolean; // True if record already existed in database
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export interface RecordCreationPlan {
|
||||
id: string;
|
||||
records: PlannedRecord[];
|
||||
executionOrder: string[]; // Ordered list of planned record IDs
|
||||
status: 'building' | 'incomplete' | 'ready' | 'executing' | 'completed' | 'failed';
|
||||
createdRecords: any[];
|
||||
errors: string[];
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// State Types
|
||||
// ============================================
|
||||
|
||||
export interface AiAssistantState {
|
||||
message: string;
|
||||
messages?: any[]; // BaseMessage[] from langchain - used when invoked by Deep Agent
|
||||
history?: AiChatMessage[];
|
||||
context: AiChatContext;
|
||||
|
||||
// Entity discovery
|
||||
systemEntities?: SystemEntities;
|
||||
|
||||
// Planning
|
||||
plan?: RecordCreationPlan;
|
||||
|
||||
// Legacy fields (kept for compatibility during transition)
|
||||
objectDefinition?: any;
|
||||
pageLayout?: any;
|
||||
extractedFields?: Record<string, any>;
|
||||
requiredFields?: string[];
|
||||
missingFields?: string[];
|
||||
action?: AiAssistantReply['action'];
|
||||
record?: any;
|
||||
reply?: string;
|
||||
}
|
||||
36
backend/src/ai-assistant/dto/ai-chat.dto.ts
Normal file
36
backend/src/ai-assistant/dto/ai-chat.dto.ts
Normal file
@@ -0,0 +1,36 @@
|
||||
import { Type } from 'class-transformer';
|
||||
import { IsNotEmpty, IsObject, IsOptional, IsString, ValidateNested } from 'class-validator';
|
||||
import { AiChatMessageDto } from './ai-chat.message.dto';
|
||||
|
||||
export class AiChatContextDto {
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
objectApiName?: string;
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
view?: string;
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
recordId?: string;
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
route?: string;
|
||||
}
|
||||
|
||||
export class AiChatRequestDto {
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
message: string;
|
||||
|
||||
@IsOptional()
|
||||
@IsObject()
|
||||
context?: AiChatContextDto;
|
||||
|
||||
@IsOptional()
|
||||
@ValidateNested({ each: true })
|
||||
@Type(() => AiChatMessageDto)
|
||||
history?: AiChatMessageDto[];
|
||||
}
|
||||
10
backend/src/ai-assistant/dto/ai-chat.message.dto.ts
Normal file
10
backend/src/ai-assistant/dto/ai-chat.message.dto.ts
Normal file
@@ -0,0 +1,10 @@
|
||||
import { IsIn, IsNotEmpty, IsString } from 'class-validator';
|
||||
|
||||
export class AiChatMessageDto {
|
||||
@IsIn(['user', 'assistant'])
|
||||
role: 'user' | 'assistant';
|
||||
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
text: string;
|
||||
}
|
||||
22
backend/src/ai-assistant/dto/ai-search.dto.ts
Normal file
22
backend/src/ai-assistant/dto/ai-search.dto.ts
Normal file
@@ -0,0 +1,22 @@
|
||||
import { Type } from 'class-transformer';
|
||||
import { IsNotEmpty, IsOptional, IsString, IsNumber } from 'class-validator';
|
||||
|
||||
export class AiSearchRequestDto {
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
objectApiName: string;
|
||||
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
query: string;
|
||||
|
||||
@IsOptional()
|
||||
@Type(() => Number)
|
||||
@IsNumber()
|
||||
page?: number;
|
||||
|
||||
@IsOptional()
|
||||
@Type(() => Number)
|
||||
@IsNumber()
|
||||
pageSize?: number;
|
||||
}
|
||||
@@ -10,14 +10,14 @@ export class AppBuilderService {
|
||||
|
||||
// Runtime endpoints
|
||||
async getApps(tenantId: string, userId: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
// For now, return all apps
|
||||
// In production, you'd filter by user permissions
|
||||
return App.query(knex).withGraphFetched('pages').orderBy('label', 'asc');
|
||||
}
|
||||
|
||||
async getApp(tenantId: string, slug: string, userId: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
const app = await App.query(knex)
|
||||
.findOne({ slug })
|
||||
.withGraphFetched('pages');
|
||||
@@ -35,7 +35,7 @@ export class AppBuilderService {
|
||||
pageSlug: string,
|
||||
userId: string,
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
const app = await this.getApp(tenantId, appSlug, userId);
|
||||
|
||||
const page = await AppPage.query(knex).findOne({
|
||||
@@ -52,12 +52,12 @@ export class AppBuilderService {
|
||||
|
||||
// Setup endpoints
|
||||
async getAllApps(tenantId: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
return App.query(knex).withGraphFetched('pages').orderBy('label', 'asc');
|
||||
}
|
||||
|
||||
async getAppForSetup(tenantId: string, slug: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
const app = await App.query(knex)
|
||||
.findOne({ slug })
|
||||
.withGraphFetched('pages');
|
||||
@@ -77,7 +77,7 @@ export class AppBuilderService {
|
||||
description?: string;
|
||||
},
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
return App.query(knex).insert({
|
||||
...data,
|
||||
displayOrder: 0,
|
||||
@@ -92,7 +92,7 @@ export class AppBuilderService {
|
||||
description?: string;
|
||||
},
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
const app = await this.getAppForSetup(tenantId, slug);
|
||||
|
||||
return App.query(knex).patchAndFetchById(app.id, data);
|
||||
@@ -109,7 +109,7 @@ export class AppBuilderService {
|
||||
sortOrder?: number;
|
||||
},
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
const app = await this.getAppForSetup(tenantId, appSlug);
|
||||
|
||||
return AppPage.query(knex).insert({
|
||||
@@ -133,7 +133,7 @@ export class AppBuilderService {
|
||||
sortOrder?: number;
|
||||
},
|
||||
) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
const app = await this.getAppForSetup(tenantId, appSlug);
|
||||
|
||||
const page = await AppPage.query(knex).findOne({
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { ConfigModule } from '@nestjs/config';
|
||||
import { BullModule } from '@nestjs/bullmq';
|
||||
import { PrismaModule } from './prisma/prisma.module';
|
||||
import { TenantModule } from './tenant/tenant.module';
|
||||
import { AuthModule } from './auth/auth.module';
|
||||
@@ -7,12 +8,22 @@ import { RbacModule } from './rbac/rbac.module';
|
||||
import { ObjectModule } from './object/object.module';
|
||||
import { AppBuilderModule } from './app-builder/app-builder.module';
|
||||
import { PageLayoutModule } from './page-layout/page-layout.module';
|
||||
import { VoiceModule } from './voice/voice.module';
|
||||
import { AiAssistantModule } from './ai-assistant/ai-assistant.module';
|
||||
import { SavedListViewModule } from './saved-list-view/saved-list-view.module';
|
||||
import { KnowledgeModule } from './knowledge/knowledge.module';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
ConfigModule.forRoot({
|
||||
isGlobal: true,
|
||||
}),
|
||||
BullModule.forRoot({
|
||||
connection: {
|
||||
host: process.env.REDIS_HOST || 'platform-redis',
|
||||
port: parseInt(process.env.REDIS_PORT || '6379', 10),
|
||||
},
|
||||
}),
|
||||
PrismaModule,
|
||||
TenantModule,
|
||||
AuthModule,
|
||||
@@ -20,6 +31,10 @@ import { PageLayoutModule } from './page-layout/page-layout.module';
|
||||
ObjectModule,
|
||||
AppBuilderModule,
|
||||
PageLayoutModule,
|
||||
VoiceModule,
|
||||
AiAssistantModule,
|
||||
SavedListViewModule,
|
||||
KnowledgeModule,
|
||||
],
|
||||
})
|
||||
export class AppModule {}
|
||||
|
||||
@@ -1,14 +1,19 @@
|
||||
import {
|
||||
Controller,
|
||||
Post,
|
||||
Get,
|
||||
Body,
|
||||
UnauthorizedException,
|
||||
HttpCode,
|
||||
HttpStatus,
|
||||
Req,
|
||||
UseGuards,
|
||||
} from '@nestjs/common';
|
||||
import { IsEmail, IsString, MinLength, IsOptional } from 'class-validator';
|
||||
import { AuthService } from './auth.service';
|
||||
import { TenantId } from '../tenant/tenant.decorator';
|
||||
import { JwtAuthGuard } from './jwt-auth.guard';
|
||||
import { CurrentUser } from './current-user.decorator';
|
||||
|
||||
class LoginDto {
|
||||
@IsEmail()
|
||||
@@ -40,17 +45,33 @@ class RegisterDto {
|
||||
export class AuthController {
|
||||
constructor(private authService: AuthService) {}
|
||||
|
||||
private isCentralSubdomain(subdomain: string): boolean {
|
||||
const centralSubdomains = (process.env.CENTRAL_SUBDOMAINS || 'central,admin').split(',');
|
||||
return centralSubdomains.includes(subdomain);
|
||||
}
|
||||
|
||||
@HttpCode(HttpStatus.OK)
|
||||
@Post('login')
|
||||
async login(@TenantId() tenantId: string, @Body() loginDto: LoginDto) {
|
||||
async login(
|
||||
@TenantId() tenantId: string,
|
||||
@Body() loginDto: LoginDto,
|
||||
@Req() req: any,
|
||||
) {
|
||||
const subdomain = req.raw?.subdomain;
|
||||
|
||||
|
||||
// If it's a central subdomain, tenantId is not required
|
||||
if (!subdomain || !this.isCentralSubdomain(subdomain)) {
|
||||
if (!tenantId) {
|
||||
throw new UnauthorizedException('Tenant ID is required');
|
||||
}
|
||||
}
|
||||
|
||||
const user = await this.authService.validateUser(
|
||||
tenantId,
|
||||
loginDto.email,
|
||||
loginDto.password,
|
||||
subdomain,
|
||||
);
|
||||
|
||||
if (!user) {
|
||||
@@ -64,10 +85,16 @@ export class AuthController {
|
||||
async register(
|
||||
@TenantId() tenantId: string,
|
||||
@Body() registerDto: RegisterDto,
|
||||
@Req() req: any,
|
||||
) {
|
||||
const subdomain = req.raw?.subdomain;
|
||||
|
||||
// If it's a central subdomain, tenantId is not required
|
||||
if (!subdomain || !this.isCentralSubdomain(subdomain)) {
|
||||
if (!tenantId) {
|
||||
throw new UnauthorizedException('Tenant ID is required');
|
||||
}
|
||||
}
|
||||
|
||||
const user = await this.authService.register(
|
||||
tenantId,
|
||||
@@ -75,6 +102,7 @@ export class AuthController {
|
||||
registerDto.password,
|
||||
registerDto.firstName,
|
||||
registerDto.lastName,
|
||||
subdomain,
|
||||
);
|
||||
|
||||
return user;
|
||||
@@ -87,4 +115,15 @@ export class AuthController {
|
||||
// This endpoint exists for consistency and potential future enhancements
|
||||
return { message: 'Logged out successfully' };
|
||||
}
|
||||
|
||||
@UseGuards(JwtAuthGuard)
|
||||
@Get('me')
|
||||
async me(@CurrentUser() user: any, @TenantId() tenantId: string) {
|
||||
// Return the current authenticated user info
|
||||
return {
|
||||
id: user.userId,
|
||||
email: user.email,
|
||||
tenantId: tenantId || user.tenantId,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import { JwtService } from '@nestjs/jwt';
|
||||
import { TenantDatabaseService } from '../tenant/tenant-database.service';
|
||||
import { getCentralPrisma } from '../prisma/central-prisma.service';
|
||||
import * as bcrypt from 'bcrypt';
|
||||
|
||||
@Injectable()
|
||||
@@ -10,12 +11,25 @@ export class AuthService {
|
||||
private jwtService: JwtService,
|
||||
) {}
|
||||
|
||||
private isCentralSubdomain(subdomain: string): boolean {
|
||||
const centralSubdomains = (process.env.CENTRAL_SUBDOMAINS || 'central,admin').split(',');
|
||||
return centralSubdomains.includes(subdomain);
|
||||
}
|
||||
|
||||
async validateUser(
|
||||
tenantId: string,
|
||||
email: string,
|
||||
password: string,
|
||||
subdomain?: string,
|
||||
): Promise<any> {
|
||||
const tenantDb = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
|
||||
// Check if this is a central subdomain
|
||||
if (subdomain && this.isCentralSubdomain(subdomain)) {
|
||||
return this.validateCentralUser(email, password);
|
||||
}
|
||||
|
||||
// Otherwise, validate as tenant user
|
||||
const tenantDb = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
const user = await tenantDb('users')
|
||||
.where({ email })
|
||||
@@ -43,6 +57,31 @@ export class AuthService {
|
||||
return null;
|
||||
}
|
||||
|
||||
private async validateCentralUser(
|
||||
email: string,
|
||||
password: string,
|
||||
): Promise<any> {
|
||||
const centralPrisma = getCentralPrisma();
|
||||
|
||||
const user = await centralPrisma.user.findUnique({
|
||||
where: { email },
|
||||
});
|
||||
|
||||
if (!user) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (await bcrypt.compare(password, user.password)) {
|
||||
const { password: _, ...result } = user;
|
||||
return {
|
||||
...result,
|
||||
isCentralAdmin: true,
|
||||
};
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
async login(user: any) {
|
||||
const payload = {
|
||||
sub: user.id,
|
||||
@@ -66,8 +105,15 @@ export class AuthService {
|
||||
password: string,
|
||||
firstName?: string,
|
||||
lastName?: string,
|
||||
subdomain?: string,
|
||||
) {
|
||||
const tenantDb = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
// Check if this is a central subdomain
|
||||
if (subdomain && this.isCentralSubdomain(subdomain)) {
|
||||
return this.registerCentralUser(email, password, firstName, lastName);
|
||||
}
|
||||
|
||||
// Otherwise, register as tenant user
|
||||
const tenantDb = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
const hashedPassword = await bcrypt.hash(password, 10);
|
||||
|
||||
@@ -88,4 +134,28 @@ export class AuthService {
|
||||
const { password: _, ...result } = user;
|
||||
return result;
|
||||
}
|
||||
|
||||
private async registerCentralUser(
|
||||
email: string,
|
||||
password: string,
|
||||
firstName?: string,
|
||||
lastName?: string,
|
||||
) {
|
||||
const centralPrisma = getCentralPrisma();
|
||||
|
||||
const hashedPassword = await bcrypt.hash(password, 10);
|
||||
|
||||
const user = await centralPrisma.user.create({
|
||||
data: {
|
||||
email,
|
||||
password: hashedPassword,
|
||||
firstName: firstName || null,
|
||||
lastName: lastName || null,
|
||||
isActive: true,
|
||||
},
|
||||
});
|
||||
|
||||
const { password: _, ...result } = user;
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
export type SemanticProjectionInput = {
|
||||
objectApiName: string;
|
||||
record: Record<string, any>;
|
||||
objectDefinition?: any;
|
||||
comments: Array<{ id: string; content: string; author_user_id: string; created_at?: string }>;
|
||||
};
|
||||
|
||||
export type SemanticProjection = {
|
||||
entityType: string;
|
||||
entityId: string;
|
||||
title: string;
|
||||
narrative: string;
|
||||
/** Plain text used for embedding — no 'key: value' labels, no comments (chunker handles those separately). */
|
||||
embeddingNarrative: string;
|
||||
metadata: Record<string, any>;
|
||||
sourceSummary: {
|
||||
includedFieldCount: number;
|
||||
includedCommentCount: number;
|
||||
includesComments: boolean;
|
||||
};
|
||||
};
|
||||
|
||||
export interface SemanticProjectionAdapter {
|
||||
supports(objectApiName: string): boolean;
|
||||
buildProjection(input: SemanticProjectionInput): SemanticProjection;
|
||||
}
|
||||
|
||||
const EXCLUDED_FIELDS = new Set([
|
||||
'id',
|
||||
'created_at',
|
||||
'updated_at',
|
||||
'ownerId',
|
||||
'owner_id',
|
||||
'tenantId',
|
||||
'tenant_id',
|
||||
]);
|
||||
|
||||
export class DefaultSemanticProjectionAdapter implements SemanticProjectionAdapter {
|
||||
supports(): boolean {
|
||||
return true;
|
||||
}
|
||||
|
||||
buildProjection(input: SemanticProjectionInput): SemanticProjection {
|
||||
const fieldEntries = Object.entries(input.record || {}).filter(([key, value]) => {
|
||||
if (EXCLUDED_FIELDS.has(key)) return false;
|
||||
if (value === null || value === undefined || value === '') return false;
|
||||
return ['string', 'number', 'boolean'].includes(typeof value);
|
||||
});
|
||||
|
||||
const title =
|
||||
input.record?.name ||
|
||||
input.record?.title ||
|
||||
input.record?.subject ||
|
||||
`${input.objectApiName} ${input.record?.id || ''}`.trim();
|
||||
|
||||
|
||||
const fieldNarrative = fieldEntries
|
||||
.map(([key, value]) => `${key}: ${String(value)}`)
|
||||
.join('\n');
|
||||
|
||||
const commentNarrative = (input.comments || [])
|
||||
.map((comment, index) => `Comment ${index + 1}: ${comment.content}`)
|
||||
.join('\n');
|
||||
|
||||
const narrative = [fieldNarrative, commentNarrative].filter(Boolean).join('\n\n');
|
||||
|
||||
// Plain values only — no 'key:' prefixes. Comments are handled separately by the chunker.
|
||||
const embeddingNarrative = fieldEntries
|
||||
.map(([, value]) => String(value))
|
||||
.join('\n');
|
||||
|
||||
return {
|
||||
entityType: input.objectApiName,
|
||||
entityId: input.record.id,
|
||||
title,
|
||||
narrative,
|
||||
embeddingNarrative,
|
||||
metadata: {
|
||||
objectApiName: input.objectApiName,
|
||||
hasComments: (input.comments || []).length > 0,
|
||||
},
|
||||
sourceSummary: {
|
||||
includedFieldCount: fieldEntries.length,
|
||||
includedCommentCount: (input.comments || []).length,
|
||||
includesComments: (input.comments || []).length > 0,
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
24
backend/src/knowledge/dto/comment.dto.ts
Normal file
24
backend/src/knowledge/dto/comment.dto.ts
Normal file
@@ -0,0 +1,24 @@
|
||||
import { IsNotEmpty, IsOptional, IsString, MaxLength, MinLength } from 'class-validator';
|
||||
|
||||
export class CreateCommentDto {
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
parentObjectApiName: string;
|
||||
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
parentRecordId: string;
|
||||
|
||||
@IsString()
|
||||
@MinLength(1)
|
||||
@MaxLength(10000)
|
||||
content: string;
|
||||
}
|
||||
|
||||
export class UpdateCommentDto {
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
@MinLength(1)
|
||||
@MaxLength(10000)
|
||||
content?: string;
|
||||
}
|
||||
52
backend/src/knowledge/dto/semantic-link.dto.ts
Normal file
52
backend/src/knowledge/dto/semantic-link.dto.ts
Normal file
@@ -0,0 +1,52 @@
|
||||
import { IsIn, IsNumber, IsObject, IsOptional, IsString, Max, Min } from 'class-validator';
|
||||
|
||||
export const SEMANTIC_LINK_STATUSES = ['suggested', 'approved', 'rejected', 'dismissed'] as const;
|
||||
export const SEMANTIC_LINK_ORIGINS = ['manual', 'semantic', 'llm', 'hybrid', 'rule_based'] as const;
|
||||
|
||||
export class ReviewSemanticLinkDto {
|
||||
@IsString()
|
||||
@IsIn(SEMANTIC_LINK_STATUSES)
|
||||
status: (typeof SEMANTIC_LINK_STATUSES)[number];
|
||||
}
|
||||
|
||||
export class UpsertSemanticLinkDto {
|
||||
@IsString()
|
||||
sourceEntityType: string;
|
||||
|
||||
@IsString()
|
||||
sourceEntityId: string;
|
||||
|
||||
@IsString()
|
||||
targetEntityType: string;
|
||||
|
||||
@IsString()
|
||||
targetEntityId: string;
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
linkType?: string;
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
@IsIn(SEMANTIC_LINK_STATUSES)
|
||||
status?: (typeof SEMANTIC_LINK_STATUSES)[number];
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
@IsIn(SEMANTIC_LINK_ORIGINS)
|
||||
origin?: (typeof SEMANTIC_LINK_ORIGINS)[number];
|
||||
|
||||
@IsOptional()
|
||||
@IsNumber()
|
||||
@Min(0)
|
||||
@Max(1)
|
||||
confidence?: number;
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
reason?: string;
|
||||
|
||||
@IsOptional()
|
||||
@IsObject()
|
||||
evidence?: Record<string, any>;
|
||||
}
|
||||
124
backend/src/knowledge/knowledge.controller.ts
Normal file
124
backend/src/knowledge/knowledge.controller.ts
Normal file
@@ -0,0 +1,124 @@
|
||||
import {
|
||||
Body,
|
||||
Controller,
|
||||
Delete,
|
||||
Get,
|
||||
Param,
|
||||
Patch,
|
||||
Post,
|
||||
Query,
|
||||
UseGuards,
|
||||
} from '@nestjs/common';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { CurrentUser } from '../auth/current-user.decorator';
|
||||
import { TenantId } from '../tenant/tenant.decorator';
|
||||
import { CreateCommentDto, UpdateCommentDto } from './dto/comment.dto';
|
||||
import { ReviewSemanticLinkDto } from './dto/semantic-link.dto';
|
||||
import { CommentService } from './services/comment.service';
|
||||
import { SemanticOrchestratorService } from './services/semantic-orchestrator.service';
|
||||
import { SemanticLinkService } from './services/semantic-link.service';
|
||||
import { TenantDatabaseService } from '../tenant/tenant-database.service';
|
||||
|
||||
@Controller('knowledge')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
export class KnowledgeController {
|
||||
constructor(
|
||||
private readonly commentService: CommentService,
|
||||
private readonly semanticOrchestratorService: SemanticOrchestratorService,
|
||||
private readonly semanticLinkService: SemanticLinkService,
|
||||
private readonly tenantDbService: TenantDatabaseService,
|
||||
) {}
|
||||
|
||||
@Get('comments/:objectApiName/:recordId')
|
||||
async getComments(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Param('recordId') recordId: string,
|
||||
) {
|
||||
return this.commentService.listComments(tenantId, objectApiName, recordId);
|
||||
}
|
||||
|
||||
@Post('comments')
|
||||
async createComment(
|
||||
@TenantId() tenantId: string,
|
||||
@Body() dto: CreateCommentDto,
|
||||
@CurrentUser() user: any,
|
||||
) {
|
||||
return this.commentService.createComment(tenantId, dto, user.userId);
|
||||
}
|
||||
|
||||
@Patch('comments/:id')
|
||||
async updateComment(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('id') id: string,
|
||||
@Body() dto: UpdateCommentDto,
|
||||
@CurrentUser() user: any,
|
||||
) {
|
||||
return this.commentService.updateComment(tenantId, id, dto, user.userId);
|
||||
}
|
||||
|
||||
@Delete('comments/:id')
|
||||
async deleteComment(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('id') id: string,
|
||||
@CurrentUser() user: any,
|
||||
) {
|
||||
return this.commentService.deleteComment(tenantId, id, user.userId);
|
||||
}
|
||||
|
||||
@Post('semantic/refresh/:objectApiName/:recordId')
|
||||
async refreshSemantic(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Param('recordId') recordId: string,
|
||||
@CurrentUser() user: any,
|
||||
) {
|
||||
return this.semanticOrchestratorService.refreshRecord(
|
||||
tenantId,
|
||||
objectApiName,
|
||||
recordId,
|
||||
user.userId,
|
||||
'manual_refresh',
|
||||
);
|
||||
}
|
||||
|
||||
@Post('semantic/reindex/:objectApiName')
|
||||
async reindexObject(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@CurrentUser() user: any,
|
||||
@Query('limit') limit?: string,
|
||||
) {
|
||||
const parsedLimit = Number.isFinite(Number(limit)) ? Number(limit) : 250;
|
||||
return this.semanticOrchestratorService.reindexObject(
|
||||
tenantId,
|
||||
objectApiName,
|
||||
user.userId,
|
||||
parsedLimit,
|
||||
);
|
||||
}
|
||||
|
||||
@Get('semantic/links/:objectApiName/:recordId')
|
||||
async listLinks(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Param('recordId') recordId: string,
|
||||
@Query('status') status?: string,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
return this.semanticLinkService.listForRecord(knex, objectApiName, recordId, status);
|
||||
}
|
||||
|
||||
@Patch('semantic/links/:id/review')
|
||||
async reviewLink(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('id') id: string,
|
||||
@Body() dto: ReviewSemanticLinkDto,
|
||||
@CurrentUser() user: any,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
return this.semanticLinkService.reviewLink(knex, id, dto.status, user.userId);
|
||||
}
|
||||
}
|
||||
31
backend/src/knowledge/knowledge.module.ts
Normal file
31
backend/src/knowledge/knowledge.module.ts
Normal file
@@ -0,0 +1,31 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { BullModule } from '@nestjs/bullmq';
|
||||
import { KnowledgeController } from './knowledge.controller';
|
||||
import { CommentService } from './services/comment.service';
|
||||
import { SemanticOrchestratorService } from './services/semantic-orchestrator.service';
|
||||
import { SemanticChunkerService } from './services/semantic-chunker.service';
|
||||
import { SemanticLinkService } from './services/semantic-link.service';
|
||||
import { SemanticRefreshQueueService } from './services/semantic-refresh-queue.service';
|
||||
import { SemanticRefreshProcessor } from './semantic-refresh.processor';
|
||||
import { TenantModule } from '../tenant/tenant.module';
|
||||
import { MeilisearchModule } from '../search/meilisearch.module';
|
||||
import { SEMANTIC_REFRESH_QUEUE } from './semantic-refresh.constants';
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
TenantModule,
|
||||
MeilisearchModule,
|
||||
BullModule.registerQueue({ name: SEMANTIC_REFRESH_QUEUE }),
|
||||
],
|
||||
controllers: [KnowledgeController],
|
||||
providers: [
|
||||
CommentService,
|
||||
SemanticOrchestratorService,
|
||||
SemanticChunkerService,
|
||||
SemanticLinkService,
|
||||
SemanticRefreshQueueService,
|
||||
SemanticRefreshProcessor,
|
||||
],
|
||||
exports: [SemanticOrchestratorService, SemanticRefreshQueueService],
|
||||
})
|
||||
export class KnowledgeModule {}
|
||||
3
backend/src/knowledge/semantic-refresh.constants.ts
Normal file
3
backend/src/knowledge/semantic-refresh.constants.ts
Normal file
@@ -0,0 +1,3 @@
|
||||
export const SEMANTIC_REFRESH_QUEUE = 'semantic-refresh';
|
||||
|
||||
export const SEMANTIC_REFRESH_JOB = 'refresh-record';
|
||||
45
backend/src/knowledge/semantic-refresh.processor.ts
Normal file
45
backend/src/knowledge/semantic-refresh.processor.ts
Normal file
@@ -0,0 +1,45 @@
|
||||
import { Processor, WorkerHost } from '@nestjs/bullmq';
|
||||
import { Logger } from '@nestjs/common';
|
||||
import { Job } from 'bullmq';
|
||||
import { SemanticOrchestratorService } from './services/semantic-orchestrator.service';
|
||||
import { SEMANTIC_REFRESH_QUEUE } from './semantic-refresh.constants';
|
||||
|
||||
export type SemanticRefreshJobData = {
|
||||
tenantId: string;
|
||||
objectApiName: string;
|
||||
recordId: string;
|
||||
userId?: string;
|
||||
trigger: string;
|
||||
};
|
||||
|
||||
@Processor(SEMANTIC_REFRESH_QUEUE)
|
||||
export class SemanticRefreshProcessor extends WorkerHost {
|
||||
private readonly logger = new Logger(SemanticRefreshProcessor.name);
|
||||
|
||||
constructor(
|
||||
private readonly semanticOrchestratorService: SemanticOrchestratorService,
|
||||
) {
|
||||
super();
|
||||
}
|
||||
|
||||
async process(job: Job<SemanticRefreshJobData>): Promise<void> {
|
||||
const { tenantId, objectApiName, recordId, userId, trigger } = job.data;
|
||||
this.logger.log(
|
||||
`Processing semantic refresh: ${objectApiName}:${recordId} trigger=${trigger}`,
|
||||
);
|
||||
try {
|
||||
await this.semanticOrchestratorService.refreshRecord(
|
||||
tenantId,
|
||||
objectApiName,
|
||||
recordId,
|
||||
userId,
|
||||
trigger,
|
||||
);
|
||||
} catch (error) {
|
||||
this.logger.error(
|
||||
`Semantic refresh failed: ${objectApiName}:${recordId} trigger=${trigger} error=${error.message}`,
|
||||
);
|
||||
throw error; // Let BullMQ handle retries
|
||||
}
|
||||
}
|
||||
}
|
||||
115
backend/src/knowledge/services/comment.service.ts
Normal file
115
backend/src/knowledge/services/comment.service.ts
Normal file
@@ -0,0 +1,115 @@
|
||||
import { ForbiddenException, Injectable, NotFoundException } from '@nestjs/common';
|
||||
import { TenantDatabaseService } from '../../tenant/tenant-database.service';
|
||||
import { CreateCommentDto, UpdateCommentDto } from '../dto/comment.dto';
|
||||
import { SemanticRefreshQueueService } from './semantic-refresh-queue.service';
|
||||
|
||||
@Injectable()
|
||||
export class CommentService {
|
||||
constructor(
|
||||
private readonly tenantDbService: TenantDatabaseService,
|
||||
private readonly semanticRefreshQueue: SemanticRefreshQueueService,
|
||||
) {}
|
||||
|
||||
async listComments(tenantId: string, parentObjectApiName: string, parentRecordId: string) {
|
||||
const knex = await this.getKnex(tenantId);
|
||||
return knex('comments')
|
||||
.where({
|
||||
parent_object_api_name: parentObjectApiName,
|
||||
parent_record_id: parentRecordId,
|
||||
})
|
||||
.orderBy('created_at', 'desc');
|
||||
}
|
||||
|
||||
async createComment(tenantId: string, dto: CreateCommentDto, userId: string) {
|
||||
const knex = await this.getKnex(tenantId);
|
||||
const [created] = await knex('comments')
|
||||
.insert({
|
||||
parent_object_api_name: dto.parentObjectApiName,
|
||||
parent_record_id: dto.parentRecordId,
|
||||
author_user_id: userId,
|
||||
content: dto.content,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
})
|
||||
.returning('*');
|
||||
|
||||
console.log(
|
||||
`[Knowledge] Comment created: ${dto.parentObjectApiName}:${dto.parentRecordId} by ${userId}`,
|
||||
);
|
||||
await this.semanticRefreshQueue.enqueue(
|
||||
tenantId,
|
||||
dto.parentObjectApiName,
|
||||
dto.parentRecordId,
|
||||
userId,
|
||||
'comment_created',
|
||||
);
|
||||
|
||||
return created;
|
||||
}
|
||||
|
||||
async updateComment(tenantId: string, commentId: string, dto: UpdateCommentDto, userId: string) {
|
||||
const knex = await this.getKnex(tenantId);
|
||||
const existing = await knex('comments').where({ id: commentId }).first();
|
||||
|
||||
if (!existing) {
|
||||
throw new NotFoundException('Comment not found');
|
||||
}
|
||||
|
||||
if (existing.author_user_id !== userId) {
|
||||
throw new ForbiddenException('Only the author can edit this comment');
|
||||
}
|
||||
|
||||
await knex('comments')
|
||||
.where({ id: commentId })
|
||||
.update({
|
||||
...(dto.content ? { content: dto.content } : {}),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
console.log(
|
||||
`[Knowledge] Comment updated: ${existing.parent_object_api_name}:${existing.parent_record_id} by ${userId}`,
|
||||
);
|
||||
await this.semanticRefreshQueue.enqueue(
|
||||
tenantId,
|
||||
existing.parent_object_api_name,
|
||||
existing.parent_record_id,
|
||||
userId,
|
||||
'comment_updated',
|
||||
);
|
||||
|
||||
return knex('comments').where({ id: commentId }).first();
|
||||
}
|
||||
|
||||
async deleteComment(tenantId: string, commentId: string, userId: string) {
|
||||
const knex = await this.getKnex(tenantId);
|
||||
const existing = await knex('comments').where({ id: commentId }).first();
|
||||
|
||||
if (!existing) {
|
||||
throw new NotFoundException('Comment not found');
|
||||
}
|
||||
|
||||
if (existing.author_user_id !== userId) {
|
||||
throw new ForbiddenException('Only the author can delete this comment');
|
||||
}
|
||||
|
||||
await knex('comments').where({ id: commentId }).delete();
|
||||
|
||||
console.log(
|
||||
`[Knowledge] Comment deleted: ${existing.parent_object_api_name}:${existing.parent_record_id} by ${userId}`,
|
||||
);
|
||||
await this.semanticRefreshQueue.enqueue(
|
||||
tenantId,
|
||||
existing.parent_object_api_name,
|
||||
existing.parent_record_id,
|
||||
userId,
|
||||
'comment_deleted',
|
||||
);
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
private async getKnex(tenantId: string) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
return this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
import { SemanticChunkerService } from './semantic-chunker.service';
|
||||
|
||||
describe('SemanticChunkerService', () => {
|
||||
let service: SemanticChunkerService;
|
||||
|
||||
beforeEach(() => {
|
||||
service = new SemanticChunkerService();
|
||||
});
|
||||
|
||||
it('creates chunks from base narrative and comments', () => {
|
||||
const chunks = service.chunkText('Intro paragraph\n\nSecond paragraph', [
|
||||
{ id: 'c-1', content: 'Comment body' },
|
||||
]);
|
||||
|
||||
expect(chunks).toHaveLength(3);
|
||||
expect(chunks[0].sourceKind).toBe('base_record');
|
||||
expect(chunks[2].sourceKind).toBe('comment');
|
||||
expect(chunks[2].sourceRefId).toBe('c-1');
|
||||
});
|
||||
});
|
||||
71
backend/src/knowledge/services/semantic-chunker.service.ts
Normal file
71
backend/src/knowledge/services/semantic-chunker.service.ts
Normal file
@@ -0,0 +1,71 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
|
||||
export type SemanticChunk = {
|
||||
chunkIndex: number;
|
||||
sourceKind: 'base_record' | 'comment' | 'mixed';
|
||||
sourceRefId: string | null;
|
||||
text: string;
|
||||
metadata: Record<string, any>;
|
||||
};
|
||||
|
||||
@Injectable()
|
||||
export class SemanticChunkerService {
|
||||
chunkText(
|
||||
baseNarrative: string,
|
||||
comments: Array<{ id: string; content: string }>,
|
||||
): SemanticChunk[] {
|
||||
const chunks: SemanticChunk[] = [];
|
||||
|
||||
const baseParts = this.splitText(baseNarrative);
|
||||
for (const [index, text] of baseParts.entries()) {
|
||||
chunks.push({
|
||||
chunkIndex: chunks.length,
|
||||
sourceKind: 'base_record',
|
||||
sourceRefId: null,
|
||||
text,
|
||||
metadata: { section: 'base', localIndex: index },
|
||||
});
|
||||
}
|
||||
|
||||
for (const comment of comments || []) {
|
||||
const commentParts = this.splitText(comment.content);
|
||||
for (const [index, text] of commentParts.entries()) {
|
||||
chunks.push({
|
||||
chunkIndex: chunks.length,
|
||||
sourceKind: 'comment',
|
||||
sourceRefId: comment.id,
|
||||
text,
|
||||
metadata: { section: 'comment', localIndex: index, commentId: comment.id },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return chunks;
|
||||
}
|
||||
|
||||
private splitText(text: string): string[] {
|
||||
const normalized = (text || '').trim();
|
||||
if (!normalized) return [];
|
||||
|
||||
const paragraphs = normalized
|
||||
.split(/\n{2,}/)
|
||||
.map((part) => part.trim())
|
||||
.filter(Boolean);
|
||||
|
||||
const chunks: string[] = [];
|
||||
for (const paragraph of paragraphs) {
|
||||
if (paragraph.length <= 500) {
|
||||
chunks.push(paragraph);
|
||||
continue;
|
||||
}
|
||||
|
||||
let cursor = 0;
|
||||
while (cursor < paragraph.length) {
|
||||
chunks.push(paragraph.slice(cursor, cursor + 500).trim());
|
||||
cursor += 500;
|
||||
}
|
||||
}
|
||||
|
||||
return chunks.filter(Boolean);
|
||||
}
|
||||
}
|
||||
20
backend/src/knowledge/services/semantic-link.service.spec.ts
Normal file
20
backend/src/knowledge/services/semantic-link.service.spec.ts
Normal file
@@ -0,0 +1,20 @@
|
||||
import { SemanticLinkService } from './semantic-link.service';
|
||||
|
||||
describe('SemanticLinkService', () => {
|
||||
let service: SemanticLinkService;
|
||||
|
||||
beforeEach(() => {
|
||||
service = new SemanticLinkService();
|
||||
});
|
||||
|
||||
it('normalizes undirected pairs in deterministic order', () => {
|
||||
const normalized = service.normalizeUndirectedPair('Contact', 'b-id', 'Account', 'a-id');
|
||||
|
||||
expect(normalized).toEqual({
|
||||
sourceEntityType: 'Account',
|
||||
sourceEntityId: 'a-id',
|
||||
targetEntityType: 'Contact',
|
||||
targetEntityId: 'b-id',
|
||||
});
|
||||
});
|
||||
});
|
||||
186
backend/src/knowledge/services/semantic-link.service.ts
Normal file
186
backend/src/knowledge/services/semantic-link.service.ts
Normal file
@@ -0,0 +1,186 @@
|
||||
import { Injectable, NotFoundException } from '@nestjs/common';
|
||||
|
||||
export type SemanticLinkUpsertInput = {
|
||||
sourceEntityType: string;
|
||||
sourceEntityId: string;
|
||||
targetEntityType: string;
|
||||
targetEntityId: string;
|
||||
linkType?: string;
|
||||
status?: string;
|
||||
origin?: string;
|
||||
confidence?: number;
|
||||
reason?: string;
|
||||
evidence?: Record<string, any>;
|
||||
suggestedByUserId?: string | null;
|
||||
};
|
||||
|
||||
@Injectable()
|
||||
export class SemanticLinkService {
|
||||
normalizeUndirectedPair(
|
||||
sourceEntityType: string,
|
||||
sourceEntityId: string,
|
||||
targetEntityType: string,
|
||||
targetEntityId: string,
|
||||
) {
|
||||
const sourceKey = `${sourceEntityType}:${sourceEntityId}`;
|
||||
const targetKey = `${targetEntityType}:${targetEntityId}`;
|
||||
|
||||
if (sourceKey <= targetKey) {
|
||||
return {
|
||||
sourceEntityType,
|
||||
sourceEntityId,
|
||||
targetEntityType,
|
||||
targetEntityId,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
sourceEntityType: targetEntityType,
|
||||
sourceEntityId: targetEntityId,
|
||||
targetEntityType: sourceEntityType,
|
||||
targetEntityId: sourceEntityId,
|
||||
};
|
||||
}
|
||||
|
||||
async upsertSuggestedLink(knex: any, input: SemanticLinkUpsertInput) {
|
||||
const normalized = this.normalizeUndirectedPair(
|
||||
input.sourceEntityType,
|
||||
input.sourceEntityId,
|
||||
input.targetEntityType,
|
||||
input.targetEntityId,
|
||||
);
|
||||
|
||||
const payload = {
|
||||
source_entity_type: normalized.sourceEntityType,
|
||||
source_entity_id: normalized.sourceEntityId,
|
||||
target_entity_type: normalized.targetEntityType,
|
||||
target_entity_id: normalized.targetEntityId,
|
||||
link_type: input.linkType || 'related_to',
|
||||
status: input.status || 'suggested',
|
||||
origin: input.origin || 'semantic',
|
||||
confidence: input.confidence ?? 0,
|
||||
reason: input.reason || null,
|
||||
evidence: input.evidence ? JSON.stringify(input.evidence) : null,
|
||||
suggested_by_user_id: input.suggestedByUserId || null,
|
||||
updated_at: knex.fn.now(),
|
||||
created_at: knex.fn.now(),
|
||||
};
|
||||
|
||||
await knex('semantic_links')
|
||||
.insert(payload)
|
||||
.onConflict([
|
||||
'source_entity_type',
|
||||
'source_entity_id',
|
||||
'target_entity_type',
|
||||
'target_entity_id',
|
||||
'link_type',
|
||||
])
|
||||
.merge({
|
||||
status: knex.raw("IF(status = 'approved', status, VALUES(status))"),
|
||||
origin: payload.origin,
|
||||
confidence: knex.raw('GREATEST(confidence, VALUES(confidence))'),
|
||||
reason: payload.reason,
|
||||
evidence: payload.evidence,
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
}
|
||||
|
||||
async listForRecord(knex: any, entityType: string, entityId: string, status?: string) {
|
||||
const query = knex('semantic_links')
|
||||
.where((builder: any) => {
|
||||
builder
|
||||
.where({ source_entity_type: entityType, source_entity_id: entityId })
|
||||
.orWhere({ target_entity_type: entityType, target_entity_id: entityId });
|
||||
})
|
||||
.orderBy('updated_at', 'desc');
|
||||
|
||||
if (status) {
|
||||
query.andWhere({ status });
|
||||
}
|
||||
|
||||
const links = await query;
|
||||
if (!links.length) return links;
|
||||
|
||||
const typeSet = new Set<string>();
|
||||
for (const link of links) {
|
||||
typeSet.add(link.source_entity_type);
|
||||
typeSet.add(link.target_entity_type);
|
||||
}
|
||||
|
||||
const definitions = await knex('object_definitions')
|
||||
.whereIn('apiName', Array.from(typeSet))
|
||||
.select('apiName', 'label', 'pluralLabel', 'tableName', 'fields');
|
||||
const definitionByType = new Map<string, any>(
|
||||
definitions.map((item: any) => [item.apiName, item]),
|
||||
);
|
||||
|
||||
const displayNameCache = new Map<string, string>();
|
||||
const getDisplayField = (definition: any) => {
|
||||
let fields = [];
|
||||
if (Array.isArray(definition?.fields)) {
|
||||
fields = definition.fields;
|
||||
} else if (typeof definition?.fields === 'string') {
|
||||
try {
|
||||
fields = JSON.parse(definition.fields);
|
||||
} catch {
|
||||
fields = [];
|
||||
}
|
||||
}
|
||||
if (fields.some((field: any) => field?.apiName === 'name')) return 'name';
|
||||
const textField = fields.find((field: any) =>
|
||||
['STRING', 'TEXT', 'EMAIL'].includes(String(field?.type || '').toUpperCase()),
|
||||
);
|
||||
return textField?.apiName || 'id';
|
||||
};
|
||||
|
||||
const resolveTableName = (definition: any) => {
|
||||
if (definition?.tableName) return definition.tableName;
|
||||
if (definition?.pluralLabel) {
|
||||
return String(definition.pluralLabel).toLowerCase().replace(/[^a-z0-9]+/g, '_');
|
||||
}
|
||||
return `${String(definition?.apiName || '').toLowerCase()}s`;
|
||||
};
|
||||
|
||||
const loadDisplayName = async (type: string, id: string) => {
|
||||
const cacheKey = `${type}:${id}`;
|
||||
if (displayNameCache.has(cacheKey)) return displayNameCache.get(cacheKey);
|
||||
const definition = definitionByType.get(type);
|
||||
if (!definition) {
|
||||
displayNameCache.set(cacheKey, id);
|
||||
return id;
|
||||
}
|
||||
const tableName = resolveTableName(definition);
|
||||
const displayField = getDisplayField(definition);
|
||||
const record = await knex(tableName).where({ id }).first();
|
||||
const display = record?.[displayField] ? String(record[displayField]) : id;
|
||||
displayNameCache.set(cacheKey, display);
|
||||
return display;
|
||||
};
|
||||
|
||||
for (const link of links) {
|
||||
link.source_entity_label = definitionByType.get(link.source_entity_type)?.label || link.source_entity_type;
|
||||
link.target_entity_label = definitionByType.get(link.target_entity_type)?.label || link.target_entity_type;
|
||||
link.source_entity_name = await loadDisplayName(link.source_entity_type, link.source_entity_id);
|
||||
link.target_entity_name = await loadDisplayName(link.target_entity_type, link.target_entity_id);
|
||||
}
|
||||
|
||||
return links;
|
||||
}
|
||||
|
||||
async reviewLink(knex: any, linkId: string, status: string, reviewerUserId: string) {
|
||||
const updated = await knex('semantic_links')
|
||||
.where({ id: linkId })
|
||||
.update({
|
||||
status,
|
||||
reviewed_by_user_id: reviewerUserId,
|
||||
reviewed_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
if (!updated) {
|
||||
throw new NotFoundException('Semantic link not found');
|
||||
}
|
||||
|
||||
return knex('semantic_links').where({ id: linkId }).first();
|
||||
}
|
||||
}
|
||||
540
backend/src/knowledge/services/semantic-orchestrator.service.ts
Normal file
540
backend/src/knowledge/services/semantic-orchestrator.service.ts
Normal file
@@ -0,0 +1,540 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
import { HumanMessage, SystemMessage } from '@langchain/core/messages';
|
||||
import { ChatOpenAI } from '@langchain/openai';
|
||||
import { TenantDatabaseService } from '../../tenant/tenant-database.service';
|
||||
import { MeilisearchService } from '../../search/meilisearch.service';
|
||||
import { getCentralPrisma } from '../../prisma/central-prisma.service';
|
||||
import { OpenAIConfig } from '../../voice/interfaces/integration-config.interface';
|
||||
import { randomUUID } from 'crypto';
|
||||
import {
|
||||
DefaultSemanticProjectionAdapter,
|
||||
SemanticProjectionAdapter,
|
||||
} from '../adapters/semantic-projection.adapter';
|
||||
import { SemanticChunkerService } from './semantic-chunker.service';
|
||||
import { SemanticLinkService } from './semantic-link.service';
|
||||
|
||||
@Injectable()
|
||||
export class SemanticOrchestratorService {
|
||||
private readonly logger = new Logger(SemanticOrchestratorService.name);
|
||||
private readonly adapters: SemanticProjectionAdapter[] = [new DefaultSemanticProjectionAdapter()];
|
||||
private readonly defaultEmbeddingModel =
|
||||
process.env.OPENAI_EMBEDDING_MODEL || 'text-embedding-3-small';
|
||||
private readonly semanticEmbedderName = 'default';
|
||||
private readonly MIN_CONFIDENCE_BASE = 0.7;
|
||||
private readonly MIN_CONFIDENCE_COMMENT = 0.52;
|
||||
private readonly defaultChatModel = process.env.OPENAI_CHAT_MODEL || 'gpt-4o-mini';
|
||||
|
||||
constructor(
|
||||
private readonly tenantDbService: TenantDatabaseService,
|
||||
private readonly meilisearchService: MeilisearchService,
|
||||
private readonly chunkerService: SemanticChunkerService,
|
||||
private readonly semanticLinkService: SemanticLinkService,
|
||||
) {}
|
||||
|
||||
async refreshRecord(
|
||||
tenantId: string,
|
||||
objectApiName: string,
|
||||
recordId: string,
|
||||
userId?: string,
|
||||
trigger: string = 'manual',
|
||||
) {
|
||||
this.logger.log(
|
||||
`Semantic refresh start: ${objectApiName}:${recordId} (trigger=${trigger})`,
|
||||
);
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const objectDefinition = await knex('object_definitions').where({ apiName: objectApiName }).first();
|
||||
if (!objectDefinition) {
|
||||
this.logger.warn(`Object definition ${objectApiName} not found. Skipping semantic refresh.`);
|
||||
return { skipped: true };
|
||||
}
|
||||
|
||||
const tableName = this.getTableName(objectDefinition);
|
||||
const record = await knex(tableName).where({ id: recordId }).first();
|
||||
if (!record) {
|
||||
this.logger.warn(`Record not found for semantic refresh: ${objectApiName}:${recordId}`);
|
||||
return { skipped: true };
|
||||
}
|
||||
|
||||
const comments = await knex('comments')
|
||||
.where({
|
||||
parent_object_api_name: objectApiName,
|
||||
parent_record_id: recordId,
|
||||
})
|
||||
.orderBy('created_at', 'asc');
|
||||
this.logger.log(
|
||||
`Semantic refresh source: ${objectApiName}:${recordId} comments=${comments.length}`,
|
||||
);
|
||||
|
||||
const adapter = this.adapters.find((candidate) => candidate.supports(objectApiName))!;
|
||||
const projection = adapter.buildProjection({
|
||||
objectApiName,
|
||||
record,
|
||||
objectDefinition,
|
||||
comments,
|
||||
});
|
||||
|
||||
const documentId = await this.upsertSemanticDocument(knex, projection);
|
||||
const chunks = this.chunkerService.chunkText(projection.embeddingNarrative, comments);
|
||||
this.logger.log(
|
||||
`Semantic refresh chunking: ${objectApiName}:${recordId} chunks=${chunks.length}`,
|
||||
);
|
||||
await this.replaceChunks(knex, documentId, chunks);
|
||||
|
||||
const openAiConfig = await this.getOpenAiConfig(resolvedTenantId);
|
||||
const embedderReady = await this.indexChunks(resolvedTenantId, projection, chunks, openAiConfig);
|
||||
await this.generateSuggestions(
|
||||
resolvedTenantId,
|
||||
projection,
|
||||
chunks,
|
||||
openAiConfig,
|
||||
embedderReady,
|
||||
userId,
|
||||
trigger,
|
||||
);
|
||||
|
||||
this.logger.log(
|
||||
`Semantic refresh complete: ${objectApiName}:${recordId} document=${documentId}`,
|
||||
);
|
||||
return { documentId, chunkCount: chunks.length };
|
||||
}
|
||||
|
||||
async reindexObject(tenantId: string, objectApiName: string, userId?: string, limit = 250) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
const objectDefinition = await knex('object_definitions').where({ apiName: objectApiName }).first();
|
||||
if (!objectDefinition) {
|
||||
return { total: 0, processed: 0 };
|
||||
}
|
||||
|
||||
const tableName = this.getTableName(objectDefinition);
|
||||
const records = await knex(tableName).select('id').limit(limit);
|
||||
|
||||
let processed = 0;
|
||||
for (const record of records) {
|
||||
await this.refreshRecord(resolvedTenantId, objectApiName, record.id, userId, 'batch_reindex');
|
||||
processed += 1;
|
||||
}
|
||||
|
||||
return { total: records.length, processed };
|
||||
}
|
||||
|
||||
private async upsertSemanticDocument(knex: any, projection: any): Promise<string> {
|
||||
const existing = await knex('semantic_documents')
|
||||
.where({ entity_type: projection.entityType, entity_id: projection.entityId })
|
||||
.first();
|
||||
|
||||
if (existing) {
|
||||
await knex('semantic_documents')
|
||||
.where({ id: existing.id })
|
||||
.update({
|
||||
title: projection.title,
|
||||
narrative: projection.narrative,
|
||||
metadata: JSON.stringify(projection.metadata || {}),
|
||||
source_summary: JSON.stringify(projection.sourceSummary || {}),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
return existing.id;
|
||||
}
|
||||
|
||||
const newId = randomUUID();
|
||||
const [created] = await knex('semantic_documents')
|
||||
.insert({
|
||||
id: newId,
|
||||
entity_type: projection.entityType,
|
||||
entity_id: projection.entityId,
|
||||
title: projection.title,
|
||||
narrative: projection.narrative,
|
||||
metadata: JSON.stringify(projection.metadata || {}),
|
||||
source_summary: JSON.stringify(projection.sourceSummary || {}),
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
})
|
||||
.returning('id');
|
||||
|
||||
if (created && typeof created === 'object' && created.id) {
|
||||
return created.id;
|
||||
}
|
||||
// MySQL may return a numeric insert id (often 0 for UUID PKs). Always trust the generated UUID.
|
||||
return newId;
|
||||
}
|
||||
|
||||
private async replaceChunks(knex: any, documentId: string, chunks: any[]) {
|
||||
if (!documentId) {
|
||||
this.logger.warn('Skipping chunk replace: missing semantic document id.');
|
||||
return;
|
||||
}
|
||||
await knex('semantic_chunks').where({ semantic_document_id: documentId }).delete();
|
||||
if (!chunks.length) return;
|
||||
|
||||
await knex('semantic_chunks').insert(
|
||||
chunks.map((chunk) => ({
|
||||
semantic_document_id: documentId,
|
||||
chunk_index: chunk.chunkIndex,
|
||||
source_kind: chunk.sourceKind,
|
||||
source_ref_id: chunk.sourceRefId,
|
||||
text: chunk.text,
|
||||
metadata: JSON.stringify(chunk.metadata || {}),
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
})),
|
||||
);
|
||||
}
|
||||
|
||||
private async indexChunks(
|
||||
tenantId: string,
|
||||
projection: any,
|
||||
chunks: any[],
|
||||
openAiConfig: OpenAIConfig | null,
|
||||
) {
|
||||
if (!this.meilisearchService.isEnabled()) {
|
||||
this.logger.warn('Meilisearch disabled; skipping semantic chunk indexing.');
|
||||
return false;
|
||||
}
|
||||
|
||||
const indexName = this.meilisearchService.buildSemanticChunkIndexName(tenantId);
|
||||
let embedderReady = false;
|
||||
if (openAiConfig?.apiKey) {
|
||||
embedderReady = await this.meilisearchService.ensureOpenAiEmbedder(indexName, {
|
||||
embedderName: this.semanticEmbedderName,
|
||||
apiKey: openAiConfig.apiKey,
|
||||
model: openAiConfig.embeddingModel || this.defaultEmbeddingModel,
|
||||
documentTemplate: '{{doc.title}}\n{{doc.text}}',
|
||||
});
|
||||
this.logger.log(
|
||||
`Meilisearch embedder ensured: index=${indexName} model=${openAiConfig.embeddingModel || this.defaultEmbeddingModel}`,
|
||||
);
|
||||
} else {
|
||||
this.logger.warn('OpenAI embedder not configured; semantic search will be lexical only.');
|
||||
}
|
||||
this.logger.log(`Indexing semantic chunks: index=${indexName} count=${chunks.length}`);
|
||||
await this.meilisearchService.upsertDocuments(indexName, chunks.map((chunk) => ({
|
||||
id: `${projection.entityType}_${projection.entityId}_${chunk.chunkIndex}`,
|
||||
entityType: projection.entityType,
|
||||
entityId: projection.entityId,
|
||||
title: projection.title,
|
||||
sourceKind: chunk.sourceKind,
|
||||
sourceRefId: chunk.sourceRefId,
|
||||
text: chunk.text,
|
||||
})));
|
||||
return embedderReady;
|
||||
}
|
||||
|
||||
private async generateSuggestions(
|
||||
tenantId: string,
|
||||
projection: any,
|
||||
chunks: any[],
|
||||
openAiConfig: OpenAIConfig | null,
|
||||
embedderReady: boolean,
|
||||
userId?: string,
|
||||
trigger: string = 'semantic_refresh',
|
||||
) {
|
||||
if (!this.meilisearchService.isEnabled() || !chunks.length) {
|
||||
this.logger.warn(
|
||||
`Skipping suggestion generation: meili=${this.meilisearchService.isEnabled()} chunks=${chunks.length}`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
const indexName = this.meilisearchService.buildSemanticChunkIndexName(tenantId);
|
||||
// Build query from all chunks (base record + comments), prioritising comments
|
||||
// since they carry the most distinctive semantic signal.
|
||||
const commentChunks = chunks.filter((c) => c.sourceKind === 'comment');
|
||||
const baseChunks = chunks.filter((c) => c.sourceKind !== 'comment');
|
||||
const orderedChunks = [...commentChunks, ...baseChunks];
|
||||
const queryText = orderedChunks.map((chunk) => chunk.text).join(' ').slice(0, 1200);
|
||||
this.logger.log(
|
||||
`Generating suggestions: index=${indexName} queryLen=${queryText.length} hybrid=${embedderReady}`,
|
||||
);
|
||||
const search = await this.meilisearchService.searchIndex(
|
||||
indexName,
|
||||
queryText,
|
||||
20,
|
||||
// semanticRatio:1.0 = pure vector search, no lexical component that would
|
||||
// match on shared tokens like 'name:' or 'Comment 1:' across all records.
|
||||
embedderReady ? { embedder: this.semanticEmbedderName, semanticRatio: 1.0 } : undefined,
|
||||
);
|
||||
this.logger.log(
|
||||
`Meilisearch results: index=${indexName} hits=${search.hits?.length || 0} total=${search.total}`,
|
||||
);
|
||||
|
||||
const candidates = new Map<string, { hit: any; confidence: number; rankingDetails?: any }>();
|
||||
for (const hit of search.hits || []) {
|
||||
// Skip self
|
||||
if (hit.entityId === projection.entityId) continue;
|
||||
|
||||
const confidence = hit._semanticScore ?? hit._rankingScore ?? 0;
|
||||
// Use a lower threshold for comment chunks (short, conversational text
|
||||
// naturally produces lower cosine similarity than structured field values).
|
||||
const isComment = hit.sourceKind === 'comment';
|
||||
const threshold = isComment ? this.MIN_CONFIDENCE_COMMENT : this.MIN_CONFIDENCE_BASE;
|
||||
this.logger.log(
|
||||
`Suggestion candidate: ${hit.entityType}:${hit.entityId} confidence=${confidence.toFixed(4)} kind=${hit.sourceKind || 'base'} threshold=${threshold} text="${String(hit.text || '').substring(0, 60)}"`,
|
||||
);
|
||||
|
||||
if (confidence < threshold) {
|
||||
this.logger.log(
|
||||
`Skipping low-confidence match: ${hit.entityType}:${hit.entityId} confidence=${confidence.toFixed(4)} < ${threshold} (${isComment ? 'comment' : 'base'})`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
const key = `${hit.entityType}:${hit.entityId}`;
|
||||
const existing = candidates.get(key);
|
||||
if (!existing || confidence > existing.confidence) {
|
||||
candidates.set(key, {
|
||||
hit,
|
||||
confidence,
|
||||
rankingDetails: hit._rankingScoreDetails || null,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
this.logger.log(`Filtered suggestions: ${candidates.size} passed thresholds (base=${this.MIN_CONFIDENCE_BASE}, comment=${this.MIN_CONFIDENCE_COMMENT})`);
|
||||
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
for (const [key, { hit, confidence, rankingDetails }] of candidates.entries()) {
|
||||
const [targetType, targetId] = key.split(':');
|
||||
const llmAssessment = await this.assessLinkWithLlm(
|
||||
openAiConfig,
|
||||
trigger,
|
||||
projection,
|
||||
chunks,
|
||||
hit,
|
||||
confidence,
|
||||
rankingDetails,
|
||||
);
|
||||
const reason =
|
||||
llmAssessment?.reason ||
|
||||
this.humanizeTrigger(trigger) ||
|
||||
'Suggested from semantic similarity.';
|
||||
await this.semanticLinkService.upsertSuggestedLink(knex, {
|
||||
sourceEntityType: projection.entityType,
|
||||
sourceEntityId: projection.entityId,
|
||||
targetEntityType: targetType,
|
||||
targetEntityId: targetId,
|
||||
linkType: llmAssessment?.linkType || 'related',
|
||||
status: 'suggested',
|
||||
origin: 'semantic',
|
||||
confidence,
|
||||
reason,
|
||||
evidence: this.buildEvidencePayload(
|
||||
trigger,
|
||||
chunks,
|
||||
hit,
|
||||
confidence,
|
||||
rankingDetails,
|
||||
llmAssessment,
|
||||
),
|
||||
suggestedByUserId: userId || null,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private buildEvidencePayload(
|
||||
trigger: string,
|
||||
chunks: any[],
|
||||
hit: any,
|
||||
confidence: number,
|
||||
rankingDetails: any,
|
||||
llmAssessment?: {
|
||||
reason?: string;
|
||||
explanation?: string;
|
||||
matchedSignals?: string[];
|
||||
} | null,
|
||||
) {
|
||||
return {
|
||||
trigger,
|
||||
explanation:
|
||||
llmAssessment?.explanation ||
|
||||
llmAssessment?.reason ||
|
||||
'Suggested using semantic similarity and ranked chunk evidence.',
|
||||
sourceSignals: chunks.slice(0, 2).map((chunk) => ({
|
||||
sourceKind: chunk.sourceKind,
|
||||
text: chunk.text.slice(0, 220),
|
||||
})),
|
||||
matchedSignals: llmAssessment?.matchedSignals || [],
|
||||
matchedChunks: [
|
||||
{
|
||||
sourceKind: hit.sourceKind,
|
||||
text: String(hit.text || '').slice(0, 220),
|
||||
score: confidence,
|
||||
rankingDetails: rankingDetails || null,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
private async assessLinkWithLlm(
|
||||
openAiConfig: OpenAIConfig | null,
|
||||
trigger: string,
|
||||
projection: any,
|
||||
chunks: any[],
|
||||
hit: any,
|
||||
confidence: number,
|
||||
rankingDetails: any,
|
||||
): Promise<{ linkType: string; reason?: string; explanation?: string; matchedSignals?: string[] } | null> {
|
||||
if (!openAiConfig?.apiKey) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const promptPayload = {
|
||||
trigger,
|
||||
source: {
|
||||
entityType: projection.entityType,
|
||||
title: projection.title,
|
||||
narrative: String(projection.narrative || '').slice(0, 900),
|
||||
keySignals: chunks.slice(0, 3).map((chunk) => ({
|
||||
sourceKind: chunk.sourceKind,
|
||||
text: String(chunk.text || '').slice(0, 220),
|
||||
})),
|
||||
},
|
||||
target: {
|
||||
entityType: hit.entityType,
|
||||
title: hit.title,
|
||||
sourceKind: hit.sourceKind,
|
||||
text: String(hit.text || '').slice(0, 300),
|
||||
},
|
||||
confidence,
|
||||
rankingDetails: rankingDetails || {},
|
||||
allowedLinkTypes: [
|
||||
'related',
|
||||
'supports',
|
||||
'contradicts',
|
||||
'expands',
|
||||
'duplicate_of',
|
||||
'references',
|
||||
'depends_on',
|
||||
],
|
||||
};
|
||||
|
||||
try {
|
||||
const model = new ChatOpenAI({
|
||||
apiKey: openAiConfig.apiKey,
|
||||
model: openAiConfig.model || this.defaultChatModel,
|
||||
temperature: 0.1,
|
||||
});
|
||||
|
||||
const response = await model.invoke([
|
||||
new SystemMessage(
|
||||
'Classify semantic relationship. Return valid JSON only with keys: linkType, reason, explanation, matchedSignals. linkType must be one of related|supports|contradicts|expands|duplicate_of|references|depends_on.',
|
||||
),
|
||||
new HumanMessage(JSON.stringify(promptPayload)),
|
||||
]);
|
||||
|
||||
const content = typeof response.content === 'string'
|
||||
? response.content
|
||||
: Array.isArray(response.content)
|
||||
? response.content.map((part: any) => (typeof part === 'string' ? part : part?.text || '')).join('')
|
||||
: '';
|
||||
const normalized = this.extractJsonObject(content);
|
||||
if (!normalized) return null;
|
||||
|
||||
const linkType = this.normalizeLinkType(normalized.linkType);
|
||||
return {
|
||||
linkType,
|
||||
reason: typeof normalized.reason === 'string' ? normalized.reason.trim() : undefined,
|
||||
explanation:
|
||||
typeof normalized.explanation === 'string' ? normalized.explanation.trim() : undefined,
|
||||
matchedSignals: Array.isArray(normalized.matchedSignals)
|
||||
? normalized.matchedSignals
|
||||
.map((item: any) => String(item || '').trim())
|
||||
.filter(Boolean)
|
||||
.slice(0, 3)
|
||||
: undefined,
|
||||
};
|
||||
} catch (error) {
|
||||
this.logger.warn(`Semantic LLM assessment failed: ${error.message}`);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private extractJsonObject(raw: string): Record<string, any> | null {
|
||||
if (!raw) return null;
|
||||
const trimmed = raw.trim();
|
||||
try {
|
||||
return JSON.parse(trimmed);
|
||||
} catch {
|
||||
const match = trimmed.match(/\{[\s\S]*\}/);
|
||||
if (!match) return null;
|
||||
try {
|
||||
return JSON.parse(match[0]);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private normalizeLinkType(value: any): string {
|
||||
const supported = new Set([
|
||||
'related',
|
||||
'supports',
|
||||
'contradicts',
|
||||
'expands',
|
||||
'duplicate_of',
|
||||
'references',
|
||||
'depends_on',
|
||||
]);
|
||||
const normalized = String(value || '').trim().toLowerCase();
|
||||
if (supported.has(normalized)) return normalized;
|
||||
return 'related';
|
||||
}
|
||||
|
||||
private humanizeTrigger(trigger: string): string {
|
||||
if (!trigger) return 'Suggested from semantic similarity.';
|
||||
const map: Record<string, string> = {
|
||||
comment_created: 'Suggested based on a comment added to the record.',
|
||||
comment_updated: 'Suggested based on a comment update.',
|
||||
manual_refresh: 'Suggested after a manual semantic refresh.',
|
||||
batch_reindex: 'Suggested during semantic reindexing.',
|
||||
};
|
||||
return map[trigger] || 'Suggested from semantic similarity.';
|
||||
}
|
||||
|
||||
private getTableName(objectDefinition: any): string {
|
||||
if (objectDefinition.tableName) return objectDefinition.tableName;
|
||||
|
||||
if (objectDefinition.pluralLabel) {
|
||||
return objectDefinition.pluralLabel.toLowerCase().replace(/[^a-z0-9]+/g, '_');
|
||||
}
|
||||
|
||||
return `${objectDefinition.apiName.toLowerCase()}s`;
|
||||
}
|
||||
|
||||
private async getOpenAiConfig(tenantId: string): Promise<OpenAIConfig | null> {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const centralPrisma = getCentralPrisma();
|
||||
const tenant = await centralPrisma.tenant.findUnique({
|
||||
where: { id: resolvedTenantId },
|
||||
select: { integrationsConfig: true },
|
||||
});
|
||||
|
||||
let config = tenant?.integrationsConfig
|
||||
? typeof tenant.integrationsConfig === 'string'
|
||||
? this.tenantDbService.decryptIntegrationsConfig(tenant.integrationsConfig)
|
||||
: tenant.integrationsConfig
|
||||
: null;
|
||||
|
||||
if (!config?.openai && process.env.OPENAI_API_KEY) {
|
||||
config = {
|
||||
...(config || {}),
|
||||
openai: {
|
||||
apiKey: process.env.OPENAI_API_KEY,
|
||||
embeddingModel: this.defaultEmbeddingModel,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
if (config?.openai?.apiKey) {
|
||||
return {
|
||||
apiKey: config.openai.apiKey,
|
||||
embeddingModel: config.openai.embeddingModel || this.defaultEmbeddingModel,
|
||||
};
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
import { InjectQueue } from '@nestjs/bullmq';
|
||||
import { Queue } from 'bullmq';
|
||||
import {
|
||||
SEMANTIC_REFRESH_QUEUE,
|
||||
SEMANTIC_REFRESH_JOB,
|
||||
} from '../semantic-refresh.constants';
|
||||
import { SemanticRefreshJobData } from '../semantic-refresh.processor';
|
||||
|
||||
@Injectable()
|
||||
export class SemanticRefreshQueueService {
|
||||
private readonly logger = new Logger(SemanticRefreshQueueService.name);
|
||||
|
||||
constructor(
|
||||
@InjectQueue(SEMANTIC_REFRESH_QUEUE) private readonly queue: Queue,
|
||||
) {}
|
||||
|
||||
async enqueue(
|
||||
tenantId: string,
|
||||
objectApiName: string,
|
||||
recordId: string,
|
||||
userId?: string,
|
||||
trigger: string = 'manual',
|
||||
): Promise<void> {
|
||||
const data: SemanticRefreshJobData = {
|
||||
tenantId,
|
||||
objectApiName,
|
||||
recordId,
|
||||
userId,
|
||||
trigger,
|
||||
};
|
||||
await this.queue.add(SEMANTIC_REFRESH_JOB, data, {
|
||||
attempts: 3,
|
||||
backoff: { type: 'exponential', delay: 2000 },
|
||||
removeOnComplete: 100,
|
||||
removeOnFail: 50,
|
||||
});
|
||||
this.logger.debug(
|
||||
`Enqueued semantic refresh: ${objectApiName}:${recordId} trigger=${trigger}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -3,13 +3,15 @@ import {
|
||||
FastifyAdapter,
|
||||
NestFastifyApplication,
|
||||
} from '@nestjs/platform-fastify';
|
||||
import { ValidationPipe } from '@nestjs/common';
|
||||
import { ValidationPipe, Logger } from '@nestjs/common';
|
||||
import { AppModule } from './app.module';
|
||||
import { VoiceService } from './voice/voice.service';
|
||||
import { AudioConverterService } from './voice/audio-converter.service';
|
||||
|
||||
async function bootstrap() {
|
||||
const app = await NestFactory.create<NestFastifyApplication>(
|
||||
AppModule,
|
||||
new FastifyAdapter(),
|
||||
new FastifyAdapter({ logger: true }),
|
||||
);
|
||||
|
||||
// Global validation pipe
|
||||
@@ -33,6 +35,145 @@ async function bootstrap() {
|
||||
const port = process.env.PORT || 3000;
|
||||
await app.listen(port, '0.0.0.0');
|
||||
|
||||
// After app is listening, register WebSocket handler
|
||||
const fastifyInstance = app.getHttpAdapter().getInstance();
|
||||
const logger = new Logger('MediaStreamWS');
|
||||
const voiceService = app.get(VoiceService);
|
||||
const audioConverter = app.get(AudioConverterService);
|
||||
|
||||
const WebSocketServer = require('ws').Server;
|
||||
const wss = new WebSocketServer({ noServer: true });
|
||||
|
||||
// Handle WebSocket upgrades at the server level
|
||||
const server = (fastifyInstance.server as any);
|
||||
|
||||
// Track active Media Streams connections: streamSid -> WebSocket
|
||||
const mediaStreams: Map<string, any> = new Map();
|
||||
|
||||
server.on('upgrade', (request: any, socket: any, head: any) => {
|
||||
if (request.url === '/api/voice/media-stream') {
|
||||
logger.log('=== MEDIA STREAM WEBSOCKET UPGRADE REQUEST ===');
|
||||
logger.log(`Path: ${request.url}`);
|
||||
|
||||
wss.handleUpgrade(request, socket, head, (ws: any) => {
|
||||
logger.log('=== MEDIA STREAM WEBSOCKET UPGRADED SUCCESSFULLY ===');
|
||||
handleMediaStreamSocket(ws);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
async function handleMediaStreamSocket(ws: any) {
|
||||
let streamSid: string | null = null;
|
||||
let callSid: string | null = null;
|
||||
let tenantDomain: string | null = null;
|
||||
let mediaPacketCount = 0;
|
||||
|
||||
ws.on('message', async (message: Buffer) => {
|
||||
try {
|
||||
const msg = JSON.parse(message.toString());
|
||||
|
||||
switch (msg.event) {
|
||||
case 'connected':
|
||||
logger.log('=== MEDIA STREAM EVENT: CONNECTED ===');
|
||||
logger.log(`Protocol: ${msg.protocol}`);
|
||||
logger.log(`Version: ${msg.version}`);
|
||||
break;
|
||||
|
||||
case 'start':
|
||||
streamSid = msg.streamSid;
|
||||
callSid = msg.start.callSid;
|
||||
tenantDomain = msg.start.customParameters?.tenantId || 'tenant1';
|
||||
|
||||
logger.log(`=== MEDIA STREAM EVENT: START ===`);
|
||||
logger.log(`StreamSid: ${streamSid}`);
|
||||
logger.log(`CallSid: ${callSid}`);
|
||||
logger.log(`Tenant: ${tenantDomain}`);
|
||||
logger.log(`MediaFormat: ${JSON.stringify(msg.start.mediaFormat)}`);
|
||||
|
||||
mediaStreams.set(streamSid, ws);
|
||||
logger.log(`Stored WebSocket for streamSid: ${streamSid}. Total active streams: ${mediaStreams.size}`);
|
||||
|
||||
// Initialize OpenAI Realtime connection
|
||||
logger.log(`Initializing OpenAI Realtime for call ${callSid}...`);
|
||||
try {
|
||||
await voiceService.initializeOpenAIRealtime({
|
||||
callSid,
|
||||
tenantId: tenantDomain,
|
||||
userId: msg.start.customParameters?.userId || 'system',
|
||||
});
|
||||
logger.log(`✓ OpenAI Realtime initialized for call ${callSid}`);
|
||||
} catch (error: any) {
|
||||
logger.error(`Failed to initialize OpenAI: ${error.message}`);
|
||||
}
|
||||
break;
|
||||
|
||||
case 'media':
|
||||
mediaPacketCount++;
|
||||
// Only log every 500 packets to reduce noise
|
||||
if (mediaPacketCount % 500 === 0) {
|
||||
logger.log(`Received media packet #${mediaPacketCount} for StreamSid: ${streamSid}`);
|
||||
}
|
||||
|
||||
if (!callSid || !tenantDomain) {
|
||||
logger.warn('Received media before start event');
|
||||
break;
|
||||
}
|
||||
|
||||
try {
|
||||
// Convert Twilio audio (μ-law 8kHz) to OpenAI format (PCM16 24kHz)
|
||||
const twilioAudio = msg.media.payload;
|
||||
const openaiAudio = audioConverter.twilioToOpenAI(twilioAudio);
|
||||
|
||||
// Send audio to OpenAI Realtime API
|
||||
await voiceService.sendAudioToOpenAI(callSid, openaiAudio);
|
||||
} catch (error: any) {
|
||||
logger.error(`Error processing media: ${error.message}`);
|
||||
}
|
||||
break;
|
||||
|
||||
case 'stop':
|
||||
logger.log(`=== MEDIA STREAM EVENT: STOP ===`);
|
||||
logger.log(`StreamSid: ${streamSid}`);
|
||||
logger.log(`Total media packets received: ${mediaPacketCount}`);
|
||||
|
||||
if (streamSid) {
|
||||
mediaStreams.delete(streamSid);
|
||||
logger.log(`Removed WebSocket for streamSid: ${streamSid}`);
|
||||
}
|
||||
|
||||
// Clean up OpenAI connection
|
||||
if (callSid) {
|
||||
try {
|
||||
logger.log(`Cleaning up OpenAI connection for call ${callSid}...`);
|
||||
await voiceService.cleanupOpenAIConnection(callSid);
|
||||
logger.log(`✓ OpenAI connection cleaned up`);
|
||||
} catch (error: any) {
|
||||
logger.error(`Failed to cleanup OpenAI: ${error.message}`);
|
||||
}
|
||||
}
|
||||
break;
|
||||
|
||||
default:
|
||||
logger.debug(`Unknown media stream event: ${msg.event}`);
|
||||
}
|
||||
} catch (error: any) {
|
||||
logger.error(`Error processing media stream message: ${error.message}`);
|
||||
}
|
||||
});
|
||||
|
||||
ws.on('close', () => {
|
||||
logger.log(`=== MEDIA STREAM WEBSOCKET CLOSED ===`);
|
||||
if (streamSid) {
|
||||
mediaStreams.delete(streamSid);
|
||||
}
|
||||
});
|
||||
|
||||
ws.on('error', (error: Error) => {
|
||||
logger.error(`=== MEDIA STREAM WEBSOCKET ERROR ===`);
|
||||
logger.error(`Error message: ${error.message}`);
|
||||
});
|
||||
}
|
||||
|
||||
console.log(`🚀 Application is running on: http://localhost:${port}/api`);
|
||||
}
|
||||
|
||||
|
||||
306
backend/src/migration/custom-migration.service.ts
Normal file
306
backend/src/migration/custom-migration.service.ts
Normal file
@@ -0,0 +1,306 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
import { Knex } from 'knex';
|
||||
|
||||
export interface CustomMigrationRecord {
|
||||
id: string;
|
||||
tenantId: string;
|
||||
name: string;
|
||||
description: string;
|
||||
type: 'create_table' | 'add_column' | 'alter_column' | 'add_index' | 'drop_table' | 'custom';
|
||||
sql: string;
|
||||
status: 'pending' | 'executed' | 'failed';
|
||||
executedAt?: Date;
|
||||
error?: string;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
}
|
||||
|
||||
@Injectable()
|
||||
export class CustomMigrationService {
|
||||
private readonly logger = new Logger(CustomMigrationService.name);
|
||||
|
||||
/**
|
||||
* Generate SQL to create a table with standard fields
|
||||
*/
|
||||
generateCreateTableSQL(
|
||||
tableName: string,
|
||||
fields: {
|
||||
apiName: string;
|
||||
type: string;
|
||||
isRequired?: boolean;
|
||||
isUnique?: boolean;
|
||||
defaultValue?: string;
|
||||
}[] = [],
|
||||
): string {
|
||||
// Start with standard fields
|
||||
const columns: string[] = [
|
||||
'`id` VARCHAR(36) PRIMARY KEY',
|
||||
'`ownerId` VARCHAR(36)',
|
||||
'`name` VARCHAR(255)',
|
||||
'`created_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP',
|
||||
'`updated_at` TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP',
|
||||
];
|
||||
|
||||
// Add custom fields
|
||||
for (const field of fields) {
|
||||
const column = this.fieldToColumn(field);
|
||||
columns.push(column);
|
||||
}
|
||||
|
||||
// Add foreign key and index for ownerId
|
||||
columns.push('INDEX `idx_owner` (`ownerId`)');
|
||||
|
||||
return `CREATE TABLE IF NOT EXISTS \`${tableName}\` (
|
||||
${columns.join(',\n ')}
|
||||
)`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert field definition to SQL column definition
|
||||
*/
|
||||
private fieldToColumn(field: {
|
||||
apiName: string;
|
||||
type: string;
|
||||
isRequired?: boolean;
|
||||
isUnique?: boolean;
|
||||
defaultValue?: string;
|
||||
}): string {
|
||||
const columnName = field.apiName;
|
||||
let columnDef = `\`${columnName}\``;
|
||||
|
||||
// Map field types to SQL types
|
||||
switch (field.type.toUpperCase()) {
|
||||
case 'TEXT':
|
||||
case 'STRING':
|
||||
columnDef += ' VARCHAR(255)';
|
||||
break;
|
||||
case 'LONG_TEXT':
|
||||
columnDef += ' LONGTEXT';
|
||||
break;
|
||||
case 'NUMBER':
|
||||
case 'DECIMAL':
|
||||
columnDef += ' DECIMAL(18, 2)';
|
||||
break;
|
||||
case 'INTEGER':
|
||||
columnDef += ' INT';
|
||||
break;
|
||||
case 'BOOLEAN':
|
||||
columnDef += ' BOOLEAN DEFAULT FALSE';
|
||||
break;
|
||||
case 'DATE':
|
||||
columnDef += ' DATE';
|
||||
break;
|
||||
case 'DATE_TIME':
|
||||
columnDef += ' DATETIME';
|
||||
break;
|
||||
case 'EMAIL':
|
||||
columnDef += ' VARCHAR(255)';
|
||||
break;
|
||||
case 'URL':
|
||||
columnDef += ' VARCHAR(2048)';
|
||||
break;
|
||||
case 'PHONE':
|
||||
columnDef += ' VARCHAR(20)';
|
||||
break;
|
||||
case 'CURRENCY':
|
||||
columnDef += ' DECIMAL(18, 2)';
|
||||
break;
|
||||
case 'PERCENT':
|
||||
columnDef += ' DECIMAL(5, 2)';
|
||||
break;
|
||||
case 'PICKLIST':
|
||||
case 'MULTI_PICKLIST':
|
||||
columnDef += ' VARCHAR(255)';
|
||||
break;
|
||||
case 'LOOKUP':
|
||||
case 'BELONGS_TO':
|
||||
columnDef += ' VARCHAR(36)';
|
||||
break;
|
||||
default:
|
||||
columnDef += ' VARCHAR(255)';
|
||||
}
|
||||
|
||||
// Add constraints
|
||||
if (field.isRequired) {
|
||||
columnDef += ' NOT NULL';
|
||||
} else {
|
||||
columnDef += ' NULL';
|
||||
}
|
||||
|
||||
if (field.isUnique) {
|
||||
columnDef += ' UNIQUE';
|
||||
}
|
||||
|
||||
if (field.defaultValue !== undefined && field.defaultValue !== null) {
|
||||
columnDef += ` DEFAULT '${field.defaultValue}'`;
|
||||
}
|
||||
|
||||
return columnDef;
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a custom migration record in the database
|
||||
*/
|
||||
async createMigrationRecord(
|
||||
tenantKnex: Knex,
|
||||
data: {
|
||||
tenantId: string;
|
||||
name: string;
|
||||
description: string;
|
||||
type: 'create_table' | 'add_column' | 'alter_column' | 'add_index' | 'drop_table' | 'custom';
|
||||
sql: string;
|
||||
},
|
||||
): Promise<CustomMigrationRecord> {
|
||||
// Ensure custom_migrations table exists
|
||||
await this.ensureMigrationsTable(tenantKnex);
|
||||
|
||||
const id = require('crypto').randomUUID();
|
||||
const now = new Date();
|
||||
|
||||
await tenantKnex('custom_migrations').insert({
|
||||
id,
|
||||
tenantId: data.tenantId,
|
||||
name: data.name,
|
||||
description: data.description,
|
||||
type: data.type,
|
||||
sql: data.sql,
|
||||
status: 'pending',
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
});
|
||||
|
||||
return tenantKnex('custom_migrations').where({ id }).first();
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a pending migration and update its status
|
||||
*/
|
||||
async executeMigration(
|
||||
tenantKnex: Knex,
|
||||
migrationId: string,
|
||||
): Promise<CustomMigrationRecord> {
|
||||
try {
|
||||
// Get the migration record
|
||||
const migration = await tenantKnex('custom_migrations')
|
||||
.where({ id: migrationId })
|
||||
.first();
|
||||
|
||||
if (!migration) {
|
||||
throw new Error(`Migration ${migrationId} not found`);
|
||||
}
|
||||
|
||||
if (migration.status === 'executed') {
|
||||
this.logger.log(`Migration ${migrationId} already executed`);
|
||||
return migration;
|
||||
}
|
||||
|
||||
// Execute the SQL
|
||||
this.logger.log(`Executing migration: ${migration.name}`);
|
||||
await tenantKnex.raw(migration.sql);
|
||||
|
||||
// Update status
|
||||
const now = new Date();
|
||||
await tenantKnex('custom_migrations')
|
||||
.where({ id: migrationId })
|
||||
.update({
|
||||
status: 'executed',
|
||||
executedAt: now,
|
||||
updated_at: now,
|
||||
});
|
||||
|
||||
this.logger.log(`Migration ${migration.name} executed successfully`);
|
||||
return tenantKnex('custom_migrations').where({ id: migrationId }).first();
|
||||
} catch (error) {
|
||||
this.logger.error(`Failed to execute migration ${migrationId}:`, error);
|
||||
|
||||
// Update status with error
|
||||
const now = new Date();
|
||||
await tenantKnex('custom_migrations')
|
||||
.where({ id: migrationId })
|
||||
.update({
|
||||
status: 'failed',
|
||||
error: error.message,
|
||||
updated_at: now,
|
||||
});
|
||||
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create and execute a migration in one step
|
||||
*/
|
||||
async createAndExecuteMigration(
|
||||
tenantKnex: Knex,
|
||||
tenantId: string,
|
||||
data: {
|
||||
name: string;
|
||||
description: string;
|
||||
type: 'create_table' | 'add_column' | 'alter_column' | 'add_index' | 'drop_table' | 'custom';
|
||||
sql: string;
|
||||
},
|
||||
): Promise<CustomMigrationRecord> {
|
||||
// Create the migration record
|
||||
const migration = await this.createMigrationRecord(tenantKnex, {
|
||||
tenantId,
|
||||
...data,
|
||||
});
|
||||
|
||||
// Execute it immediately
|
||||
return this.executeMigration(tenantKnex, migration.id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure the custom_migrations table exists in the tenant database
|
||||
*/
|
||||
private async ensureMigrationsTable(tenantKnex: Knex): Promise<void> {
|
||||
const hasTable = await tenantKnex.schema.hasTable('custom_migrations');
|
||||
|
||||
if (!hasTable) {
|
||||
await tenantKnex.schema.createTable('custom_migrations', (table) => {
|
||||
table.uuid('id').primary();
|
||||
table.uuid('tenantId').notNullable();
|
||||
table.string('name', 255).notNullable();
|
||||
table.text('description');
|
||||
table.enum('type', ['create_table', 'add_column', 'alter_column', 'add_index', 'drop_table', 'custom']).notNullable();
|
||||
table.text('sql').notNullable();
|
||||
table.enum('status', ['pending', 'executed', 'failed']).defaultTo('pending');
|
||||
table.timestamp('executedAt').nullable();
|
||||
table.text('error').nullable();
|
||||
table.timestamps(true, true);
|
||||
|
||||
table.index(['tenantId']);
|
||||
table.index(['status']);
|
||||
table.index(['created_at']);
|
||||
});
|
||||
|
||||
this.logger.log('Created custom_migrations table');
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all migrations for a tenant
|
||||
*/
|
||||
async getMigrations(
|
||||
tenantKnex: Knex,
|
||||
tenantId: string,
|
||||
filter?: {
|
||||
status?: 'pending' | 'executed' | 'failed';
|
||||
type?: string;
|
||||
},
|
||||
): Promise<CustomMigrationRecord[]> {
|
||||
await this.ensureMigrationsTable(tenantKnex);
|
||||
|
||||
let query = tenantKnex('custom_migrations').where({ tenantId });
|
||||
|
||||
if (filter?.status) {
|
||||
query = query.where({ status: filter.status });
|
||||
}
|
||||
|
||||
if (filter?.type) {
|
||||
query = query.where({ type: filter.type });
|
||||
}
|
||||
|
||||
return query.orderBy('created_at', 'asc');
|
||||
}
|
||||
}
|
||||
10
backend/src/migration/migration.module.ts
Normal file
10
backend/src/migration/migration.module.ts
Normal file
@@ -0,0 +1,10 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { CustomMigrationService } from './custom-migration.service';
|
||||
import { TenantModule } from '../tenant/tenant.module';
|
||||
|
||||
@Module({
|
||||
imports: [TenantModule],
|
||||
providers: [CustomMigrationService],
|
||||
exports: [CustomMigrationService],
|
||||
})
|
||||
export class MigrationModule {}
|
||||
@@ -1,7 +1,38 @@
|
||||
import { Model, ModelOptions, QueryContext, snakeCaseMappers } from 'objection';
|
||||
import { Model, ModelOptions, QueryContext } from 'objection';
|
||||
|
||||
export class BaseModel extends Model {
|
||||
static columnNameMappers = snakeCaseMappers();
|
||||
/**
|
||||
* Use a minimal column mapper: keep property names as-is, but handle
|
||||
* timestamp fields that are stored as created_at/updated_at in the DB.
|
||||
*/
|
||||
static columnNameMappers = {
|
||||
parse(dbRow: Record<string, any>) {
|
||||
const mapped: Record<string, any> = {};
|
||||
for (const [key, value] of Object.entries(dbRow || {})) {
|
||||
if (key === 'created_at') {
|
||||
mapped.createdAt = value;
|
||||
} else if (key === 'updated_at') {
|
||||
mapped.updatedAt = value;
|
||||
} else {
|
||||
mapped[key] = value;
|
||||
}
|
||||
}
|
||||
return mapped;
|
||||
},
|
||||
format(model: Record<string, any>) {
|
||||
const mapped: Record<string, any> = {};
|
||||
for (const [key, value] of Object.entries(model || {})) {
|
||||
if (key === 'createdAt') {
|
||||
mapped.created_at = value;
|
||||
} else if (key === 'updatedAt') {
|
||||
mapped.updated_at = value;
|
||||
} else {
|
||||
mapped[key] = value;
|
||||
}
|
||||
}
|
||||
return mapped;
|
||||
},
|
||||
};
|
||||
|
||||
id: string;
|
||||
createdAt: Date;
|
||||
|
||||
114
backend/src/models/central.model.ts
Normal file
114
backend/src/models/central.model.ts
Normal file
@@ -0,0 +1,114 @@
|
||||
import { Model, ModelOptions, QueryContext } from 'objection';
|
||||
import { randomUUID } from 'crypto';
|
||||
|
||||
/**
|
||||
* Central database models using Objection.js
|
||||
* These models work with the central database (not tenant databases)
|
||||
*/
|
||||
|
||||
export class CentralTenant extends Model {
|
||||
static tableName = 'tenants';
|
||||
|
||||
id: string;
|
||||
name: string;
|
||||
slug: string;
|
||||
dbHost: string;
|
||||
dbPort: number;
|
||||
dbName: string;
|
||||
dbUsername: string;
|
||||
dbPassword: string;
|
||||
status: string;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
|
||||
// Relations
|
||||
domains?: CentralDomain[];
|
||||
|
||||
$beforeInsert(queryContext: QueryContext) {
|
||||
this.id = this.id || randomUUID();
|
||||
// Auto-generate slug from name if not provided
|
||||
if (!this.slug && this.name) {
|
||||
this.slug = this.name.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '');
|
||||
}
|
||||
this.createdAt = new Date();
|
||||
this.updatedAt = new Date();
|
||||
}
|
||||
|
||||
$beforeUpdate(opt: ModelOptions, queryContext: QueryContext) {
|
||||
this.updatedAt = new Date();
|
||||
}
|
||||
|
||||
static get relationMappings() {
|
||||
return {
|
||||
domains: {
|
||||
relation: Model.HasManyRelation,
|
||||
modelClass: CentralDomain,
|
||||
join: {
|
||||
from: 'tenants.id',
|
||||
to: 'domains.tenantId',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export class CentralDomain extends Model {
|
||||
static tableName = 'domains';
|
||||
|
||||
id: string;
|
||||
domain: string;
|
||||
tenantId: string;
|
||||
isPrimary: boolean;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
|
||||
// Relations
|
||||
tenant?: CentralTenant;
|
||||
|
||||
$beforeInsert(queryContext: QueryContext) {
|
||||
this.id = this.id || randomUUID();
|
||||
this.createdAt = new Date();
|
||||
this.updatedAt = new Date();
|
||||
}
|
||||
|
||||
$beforeUpdate(opt: ModelOptions, queryContext: QueryContext) {
|
||||
this.updatedAt = new Date();
|
||||
}
|
||||
|
||||
static get relationMappings() {
|
||||
return {
|
||||
tenant: {
|
||||
relation: Model.BelongsToOneRelation,
|
||||
modelClass: CentralTenant,
|
||||
join: {
|
||||
from: 'domains.tenantId',
|
||||
to: 'tenants.id',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export class CentralUser extends Model {
|
||||
static tableName = 'users';
|
||||
|
||||
id: string;
|
||||
email: string;
|
||||
password: string;
|
||||
firstName: string | null;
|
||||
lastName: string | null;
|
||||
role: string;
|
||||
isActive: boolean;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
|
||||
$beforeInsert(queryContext: QueryContext) {
|
||||
this.id = this.id || randomUUID();
|
||||
this.createdAt = new Date();
|
||||
this.updatedAt = new Date();
|
||||
}
|
||||
|
||||
$beforeUpdate(opt: ModelOptions, queryContext: QueryContext) {
|
||||
this.updatedAt = new Date();
|
||||
}
|
||||
}
|
||||
33
backend/src/models/contact-detail.model.ts
Normal file
33
backend/src/models/contact-detail.model.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { BaseModel } from './base.model';
|
||||
|
||||
export class ContactDetail extends BaseModel {
|
||||
static tableName = 'contact_details';
|
||||
|
||||
id!: string;
|
||||
relatedObjectType!: 'Account' | 'Contact';
|
||||
relatedObjectId!: string;
|
||||
detailType!: string;
|
||||
label?: string;
|
||||
value!: string;
|
||||
isPrimary!: boolean;
|
||||
|
||||
// Provide optional relations for each supported parent type.
|
||||
static relationMappings = {
|
||||
account: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: 'account.model',
|
||||
join: {
|
||||
from: 'contact_details.relatedObjectId',
|
||||
to: 'accounts.id',
|
||||
},
|
||||
},
|
||||
contact: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: 'contact.model',
|
||||
join: {
|
||||
from: 'contact_details.relatedObjectId',
|
||||
to: 'contacts.id',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
30
backend/src/models/contact.model.ts
Normal file
30
backend/src/models/contact.model.ts
Normal file
@@ -0,0 +1,30 @@
|
||||
import { BaseModel } from './base.model';
|
||||
|
||||
export class Contact extends BaseModel {
|
||||
static tableName = 'contacts';
|
||||
|
||||
id!: string;
|
||||
firstName!: string;
|
||||
lastName!: string;
|
||||
accountId!: string;
|
||||
ownerId?: string;
|
||||
|
||||
static relationMappings = {
|
||||
account: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: 'account.model',
|
||||
join: {
|
||||
from: 'contacts.accountId',
|
||||
to: 'accounts.id',
|
||||
},
|
||||
},
|
||||
owner: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: 'user.model',
|
||||
join: {
|
||||
from: 'contacts.ownerId',
|
||||
to: 'users.id',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -30,6 +30,8 @@ export interface UIMetadata {
|
||||
step?: number; // For number
|
||||
accept?: string; // For file/image
|
||||
relationDisplayField?: string; // Which field to display for relations
|
||||
relationObjects?: string[]; // For polymorphic relations
|
||||
relationTypeField?: string; // Field API name storing the selected relation type
|
||||
|
||||
// Formatting
|
||||
format?: string; // Date format, number format, etc.
|
||||
@@ -74,5 +76,13 @@ export class FieldDefinition extends BaseModel {
|
||||
to: 'object_definitions.id',
|
||||
},
|
||||
},
|
||||
rolePermissions: {
|
||||
relation: BaseModel.HasManyRelation,
|
||||
modelClass: () => require('./role-field-permission.model').RoleFieldPermission,
|
||||
join: {
|
||||
from: 'field_definitions.id',
|
||||
to: 'role_field_permissions.fieldDefinitionId',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -10,8 +10,11 @@ export class ObjectDefinition extends BaseModel {
|
||||
description?: string;
|
||||
isSystem: boolean;
|
||||
isCustom: boolean;
|
||||
orgWideDefault: 'private' | 'public_read' | 'public_read_write';
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
fields?: any[];
|
||||
rolePermissions?: any[];
|
||||
|
||||
static get jsonSchema() {
|
||||
return {
|
||||
@@ -25,12 +28,14 @@ export class ObjectDefinition extends BaseModel {
|
||||
description: { type: 'string' },
|
||||
isSystem: { type: 'boolean' },
|
||||
isCustom: { type: 'boolean' },
|
||||
orgWideDefault: { type: 'string', enum: ['private', 'public_read', 'public_read_write'] },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
static get relationMappings() {
|
||||
const { FieldDefinition } = require('./field-definition.model');
|
||||
const { RoleObjectPermission } = require('./role-object-permission.model');
|
||||
|
||||
return {
|
||||
fields: {
|
||||
@@ -41,6 +46,14 @@ export class ObjectDefinition extends BaseModel {
|
||||
to: 'field_definitions.objectDefinitionId',
|
||||
},
|
||||
},
|
||||
rolePermissions: {
|
||||
relation: BaseModel.HasManyRelation,
|
||||
modelClass: RoleObjectPermission,
|
||||
join: {
|
||||
from: 'object_definitions.id',
|
||||
to: 'role_object_permissions.objectDefinitionId',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
113
backend/src/models/record-share.model.ts
Normal file
113
backend/src/models/record-share.model.ts
Normal file
@@ -0,0 +1,113 @@
|
||||
import { BaseModel } from './base.model';
|
||||
|
||||
export interface RecordShareAccessLevel {
|
||||
canRead: boolean;
|
||||
canEdit: boolean;
|
||||
canDelete: boolean;
|
||||
}
|
||||
|
||||
export class RecordShare extends BaseModel {
|
||||
static tableName = 'record_shares';
|
||||
|
||||
// Don't use snake_case mapping since DB columns are already camelCase
|
||||
static get columnNameMappers() {
|
||||
return {
|
||||
parse(obj: any) {
|
||||
return obj;
|
||||
},
|
||||
format(obj: any) {
|
||||
return obj;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Don't auto-set timestamps - let DB defaults handle them
|
||||
$beforeInsert() {
|
||||
// Don't call super - skip BaseModel's timestamp logic
|
||||
}
|
||||
|
||||
$beforeUpdate() {
|
||||
// Don't call super - skip BaseModel's timestamp logic
|
||||
}
|
||||
|
||||
id!: string;
|
||||
objectDefinitionId!: string;
|
||||
recordId!: string;
|
||||
granteeUserId!: string;
|
||||
grantedByUserId!: string;
|
||||
accessLevel!: RecordShareAccessLevel;
|
||||
expiresAt?: Date;
|
||||
revokedAt?: Date;
|
||||
createdAt!: Date;
|
||||
updatedAt!: Date;
|
||||
|
||||
static get jsonSchema() {
|
||||
return {
|
||||
type: 'object',
|
||||
required: ['objectDefinitionId', 'recordId', 'granteeUserId', 'grantedByUserId', 'accessLevel'],
|
||||
properties: {
|
||||
id: { type: 'string' },
|
||||
objectDefinitionId: { type: 'string' },
|
||||
recordId: { type: 'string' },
|
||||
granteeUserId: { type: 'string' },
|
||||
grantedByUserId: { type: 'string' },
|
||||
accessLevel: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
canRead: { type: 'boolean' },
|
||||
canEdit: { type: 'boolean' },
|
||||
canDelete: { type: 'boolean' },
|
||||
},
|
||||
},
|
||||
expiresAt: {
|
||||
anyOf: [
|
||||
{ type: 'string', format: 'date-time' },
|
||||
{ type: 'null' },
|
||||
{ type: 'object' } // Allow Date objects
|
||||
]
|
||||
},
|
||||
revokedAt: {
|
||||
anyOf: [
|
||||
{ type: 'string', format: 'date-time' },
|
||||
{ type: 'null' },
|
||||
{ type: 'object' } // Allow Date objects
|
||||
]
|
||||
},
|
||||
createdAt: { type: ['string', 'object'], format: 'date-time' },
|
||||
updatedAt: { type: ['string', 'object'], format: 'date-time' },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
static get relationMappings() {
|
||||
const { ObjectDefinition } = require('./object-definition.model');
|
||||
const { User } = require('./user.model');
|
||||
|
||||
return {
|
||||
objectDefinition: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: ObjectDefinition,
|
||||
join: {
|
||||
from: 'record_shares.objectDefinitionId',
|
||||
to: 'object_definitions.id',
|
||||
},
|
||||
},
|
||||
granteeUser: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: User,
|
||||
join: {
|
||||
from: 'record_shares.granteeUserId',
|
||||
to: 'users.id',
|
||||
},
|
||||
},
|
||||
grantedByUser: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: User,
|
||||
join: {
|
||||
from: 'record_shares.grantedByUserId',
|
||||
to: 'users.id',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
51
backend/src/models/role-field-permission.model.ts
Normal file
51
backend/src/models/role-field-permission.model.ts
Normal file
@@ -0,0 +1,51 @@
|
||||
import { BaseModel } from './base.model';
|
||||
|
||||
export class RoleFieldPermission extends BaseModel {
|
||||
static tableName = 'role_field_permissions';
|
||||
|
||||
id!: string;
|
||||
roleId!: string;
|
||||
fieldDefinitionId!: string;
|
||||
canRead!: boolean;
|
||||
canEdit!: boolean;
|
||||
createdAt!: Date;
|
||||
updatedAt!: Date;
|
||||
|
||||
static get jsonSchema() {
|
||||
return {
|
||||
type: 'object',
|
||||
required: ['roleId', 'fieldDefinitionId'],
|
||||
properties: {
|
||||
id: { type: 'string' },
|
||||
roleId: { type: 'string' },
|
||||
fieldDefinitionId: { type: 'string' },
|
||||
canRead: { type: 'boolean' },
|
||||
canEdit: { type: 'boolean' },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
static get relationMappings() {
|
||||
const { Role } = require('./role.model');
|
||||
const { FieldDefinition } = require('./field-definition.model');
|
||||
|
||||
return {
|
||||
role: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: Role,
|
||||
join: {
|
||||
from: 'role_field_permissions.roleId',
|
||||
to: 'roles.id',
|
||||
},
|
||||
},
|
||||
fieldDefinition: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: FieldDefinition,
|
||||
join: {
|
||||
from: 'role_field_permissions.fieldDefinitionId',
|
||||
to: 'field_definitions.id',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
59
backend/src/models/role-object-permission.model.ts
Normal file
59
backend/src/models/role-object-permission.model.ts
Normal file
@@ -0,0 +1,59 @@
|
||||
import { BaseModel } from './base.model';
|
||||
|
||||
export class RoleObjectPermission extends BaseModel {
|
||||
static tableName = 'role_object_permissions';
|
||||
|
||||
id!: string;
|
||||
roleId!: string;
|
||||
objectDefinitionId!: string;
|
||||
canCreate!: boolean;
|
||||
canRead!: boolean;
|
||||
canEdit!: boolean;
|
||||
canDelete!: boolean;
|
||||
canViewAll!: boolean;
|
||||
canModifyAll!: boolean;
|
||||
createdAt!: Date;
|
||||
updatedAt!: Date;
|
||||
|
||||
static get jsonSchema() {
|
||||
return {
|
||||
type: 'object',
|
||||
required: ['roleId', 'objectDefinitionId'],
|
||||
properties: {
|
||||
id: { type: 'string' },
|
||||
roleId: { type: 'string' },
|
||||
objectDefinitionId: { type: 'string' },
|
||||
canCreate: { type: 'boolean' },
|
||||
canRead: { type: 'boolean' },
|
||||
canEdit: { type: 'boolean' },
|
||||
canDelete: { type: 'boolean' },
|
||||
canViewAll: { type: 'boolean' },
|
||||
canModifyAll: { type: 'boolean' },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
static get relationMappings() {
|
||||
const { Role } = require('./role.model');
|
||||
const { ObjectDefinition } = require('./object-definition.model');
|
||||
|
||||
return {
|
||||
role: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: Role,
|
||||
join: {
|
||||
from: 'role_object_permissions.roleId',
|
||||
to: 'roles.id',
|
||||
},
|
||||
},
|
||||
objectDefinition: {
|
||||
relation: BaseModel.BelongsToOneRelation,
|
||||
modelClass: ObjectDefinition,
|
||||
join: {
|
||||
from: 'role_object_permissions.objectDefinitionId',
|
||||
to: 'object_definitions.id',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -27,6 +27,8 @@ export class Role extends BaseModel {
|
||||
const { RolePermission } = require('./role-permission.model');
|
||||
const { Permission } = require('./permission.model');
|
||||
const { User } = require('./user.model');
|
||||
const { RoleObjectPermission } = require('./role-object-permission.model');
|
||||
const { RoleFieldPermission } = require('./role-field-permission.model');
|
||||
|
||||
return {
|
||||
rolePermissions: {
|
||||
@@ -61,6 +63,22 @@ export class Role extends BaseModel {
|
||||
to: 'users.id',
|
||||
},
|
||||
},
|
||||
objectPermissions: {
|
||||
relation: BaseModel.HasManyRelation,
|
||||
modelClass: RoleObjectPermission,
|
||||
join: {
|
||||
from: 'roles.id',
|
||||
to: 'role_object_permissions.roleId',
|
||||
},
|
||||
},
|
||||
fieldPermissions: {
|
||||
relation: BaseModel.HasManyRelation,
|
||||
modelClass: RoleFieldPermission,
|
||||
join: {
|
||||
from: 'roles.id',
|
||||
to: 'role_field_permissions.roleId',
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { BaseModel } from './base.model';
|
||||
import { ModelOptions, QueryContext } from 'objection';
|
||||
|
||||
export class User extends BaseModel {
|
||||
static tableName = 'users';
|
||||
@@ -8,6 +9,8 @@ export class User extends BaseModel {
|
||||
password: string;
|
||||
firstName?: string;
|
||||
lastName?: string;
|
||||
alias?: string;
|
||||
name?: string;
|
||||
isActive: boolean;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
@@ -22,11 +25,37 @@ export class User extends BaseModel {
|
||||
password: { type: 'string' },
|
||||
firstName: { type: 'string' },
|
||||
lastName: { type: 'string' },
|
||||
alias: { type: 'string' },
|
||||
name: { type: 'string' },
|
||||
isActive: { type: 'boolean' },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute the `name` column before insert/update so lookup fields
|
||||
* referencing User.name always have a value.
|
||||
*/
|
||||
private computeName() {
|
||||
if (this.alias) {
|
||||
this.name = this.alias;
|
||||
} else if (this.firstName || this.lastName) {
|
||||
this.name = [this.firstName, this.lastName].filter(Boolean).join(' ');
|
||||
} else if (this.email) {
|
||||
this.name = this.email;
|
||||
}
|
||||
}
|
||||
|
||||
$beforeInsert(queryContext: QueryContext) {
|
||||
super.$beforeInsert(queryContext);
|
||||
this.computeName();
|
||||
}
|
||||
|
||||
$beforeUpdate(opt: ModelOptions, queryContext: QueryContext) {
|
||||
super.$beforeUpdate(opt, queryContext);
|
||||
this.computeName();
|
||||
}
|
||||
|
||||
static get relationMappings() {
|
||||
const { UserRole } = require('./user-role.model');
|
||||
const { Role } = require('./role.model');
|
||||
|
||||
@@ -22,7 +22,9 @@ export interface FieldConfigDTO {
|
||||
step?: number;
|
||||
accept?: string;
|
||||
relationObject?: string;
|
||||
relationObjects?: string[];
|
||||
relationDisplayField?: string;
|
||||
relationTypeField?: string;
|
||||
format?: string;
|
||||
prefix?: string;
|
||||
suffix?: string;
|
||||
@@ -43,6 +45,14 @@ export interface ObjectDefinitionDTO {
|
||||
description?: string;
|
||||
isSystem: boolean;
|
||||
fields: FieldConfigDTO[];
|
||||
relatedLists?: Array<{
|
||||
title: string;
|
||||
relationName: string;
|
||||
objectApiName: string;
|
||||
fields: FieldConfigDTO[];
|
||||
canCreate?: boolean;
|
||||
createRoute?: string;
|
||||
}>;
|
||||
}
|
||||
|
||||
@Injectable()
|
||||
@@ -51,13 +61,33 @@ export class FieldMapperService {
|
||||
* Convert a field definition from the database to a frontend-friendly FieldConfig
|
||||
*/
|
||||
mapFieldToDTO(field: any): FieldConfigDTO {
|
||||
const uiMetadata = field.uiMetadata || {};
|
||||
// Parse ui_metadata if it's a JSON string or object
|
||||
let uiMetadata: any = {};
|
||||
const metadataField = field.ui_metadata || field.uiMetadata;
|
||||
if (metadataField) {
|
||||
if (typeof metadataField === 'string') {
|
||||
try {
|
||||
uiMetadata = JSON.parse(metadataField);
|
||||
} catch (e) {
|
||||
uiMetadata = {};
|
||||
}
|
||||
} else {
|
||||
uiMetadata = metadataField;
|
||||
}
|
||||
}
|
||||
|
||||
const frontendType = this.mapFieldType(field.type);
|
||||
const isLookupField = frontendType === 'belongsTo' || field.type.toLowerCase().includes('lookup');
|
||||
|
||||
// Hide 'id' field from list view by default
|
||||
const isIdField = field.apiName === 'id';
|
||||
const defaultShowOnList = isIdField ? false : true;
|
||||
|
||||
return {
|
||||
id: field.id,
|
||||
apiName: field.apiName,
|
||||
label: field.label,
|
||||
type: this.mapFieldType(field.type),
|
||||
type: frontendType,
|
||||
|
||||
// Display properties
|
||||
placeholder: uiMetadata.placeholder || field.description,
|
||||
@@ -69,7 +99,7 @@ export class FieldMapperService {
|
||||
isReadOnly: field.isSystem || uiMetadata.isReadOnly || false,
|
||||
|
||||
// View visibility
|
||||
showOnList: uiMetadata.showOnList !== false,
|
||||
showOnList: uiMetadata.showOnList !== undefined ? uiMetadata.showOnList : defaultShowOnList,
|
||||
showOnDetail: uiMetadata.showOnDetail !== false,
|
||||
showOnEdit: uiMetadata.showOnEdit !== false && !field.isSystem,
|
||||
sortable: uiMetadata.sortable !== false,
|
||||
@@ -82,7 +112,12 @@ export class FieldMapperService {
|
||||
step: uiMetadata.step,
|
||||
accept: uiMetadata.accept,
|
||||
relationObject: field.referenceObject,
|
||||
relationDisplayField: uiMetadata.relationDisplayField,
|
||||
relationObjects: uiMetadata.relationObjects,
|
||||
// For lookup fields, provide default display field if not specified
|
||||
relationDisplayField: isLookupField
|
||||
? (uiMetadata.relationDisplayField || 'name')
|
||||
: uiMetadata.relationDisplayField,
|
||||
relationTypeField: uiMetadata.relationTypeField,
|
||||
|
||||
// Formatting
|
||||
format: uiMetadata.format,
|
||||
@@ -110,12 +145,14 @@ export class FieldMapperService {
|
||||
'boolean': 'boolean',
|
||||
'date': 'date',
|
||||
'datetime': 'datetime',
|
||||
'date_time': 'datetime',
|
||||
'time': 'time',
|
||||
'email': 'email',
|
||||
'url': 'url',
|
||||
'phone': 'text',
|
||||
'picklist': 'select',
|
||||
'multipicklist': 'multiSelect',
|
||||
'multi_picklist': 'multiSelect',
|
||||
'lookup': 'belongsTo',
|
||||
'master-detail': 'belongsTo',
|
||||
'currency': 'currency',
|
||||
@@ -187,6 +224,17 @@ export class FieldMapperService {
|
||||
.filter((f: any) => f.isActive !== false)
|
||||
.sort((a: any, b: any) => (a.displayOrder || 0) - (b.displayOrder || 0))
|
||||
.map((f: any) => this.mapFieldToDTO(f)),
|
||||
relatedLists: (objectDef.relatedLists || []).map((list: any) => ({
|
||||
title: list.title,
|
||||
relationName: list.relationName,
|
||||
objectApiName: list.objectApiName,
|
||||
fields: (list.fields || [])
|
||||
.filter((f: any) => f.isActive !== false)
|
||||
.map((f: any) => this.mapFieldToDTO(f))
|
||||
.filter((f: any) => f.showOnList !== false),
|
||||
canCreate: list.canCreate,
|
||||
createRoute: list.createRoute,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
33
backend/src/object/models/base.model.ts
Normal file
33
backend/src/object/models/base.model.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { Model } from 'objection';
|
||||
import { randomUUID } from 'crypto';
|
||||
|
||||
/**
|
||||
* Base model for all dynamic and system models
|
||||
* Provides common functionality for all objects
|
||||
*/
|
||||
export class BaseModel extends Model {
|
||||
// Common fields
|
||||
id?: string;
|
||||
tenantId?: string;
|
||||
ownerId?: string;
|
||||
name?: string;
|
||||
created_at?: string;
|
||||
updated_at?: string;
|
||||
|
||||
// Hook to set system-managed fields
|
||||
async $beforeInsert() {
|
||||
if (!this.id) {
|
||||
this.id = randomUUID();
|
||||
}
|
||||
if (!this.created_at) {
|
||||
this.created_at = new Date().toISOString().slice(0, 19).replace('T', ' ');
|
||||
}
|
||||
if (!this.updated_at) {
|
||||
this.updated_at = new Date().toISOString().slice(0, 19).replace('T', ' ');
|
||||
}
|
||||
}
|
||||
|
||||
async $beforeUpdate() {
|
||||
this.updated_at = new Date().toISOString().slice(0, 19).replace('T', ' ');
|
||||
}
|
||||
}
|
||||
258
backend/src/object/models/dynamic-model.factory.ts
Normal file
258
backend/src/object/models/dynamic-model.factory.ts
Normal file
@@ -0,0 +1,258 @@
|
||||
import { ModelClass, JSONSchema, RelationMappings, Model } from 'objection';
|
||||
import { BaseModel } from './base.model';
|
||||
|
||||
export interface FieldDefinition {
|
||||
apiName: string;
|
||||
label: string;
|
||||
type: string;
|
||||
isRequired?: boolean;
|
||||
isUnique?: boolean;
|
||||
referenceObject?: string;
|
||||
defaultValue?: string;
|
||||
}
|
||||
|
||||
export interface RelationDefinition {
|
||||
name: string;
|
||||
type: 'belongsTo' | 'hasMany' | 'hasManyThrough';
|
||||
targetObjectApiName: string;
|
||||
fromColumn: string;
|
||||
toColumn: string;
|
||||
}
|
||||
|
||||
export interface ObjectMetadata {
|
||||
apiName: string;
|
||||
tableName: string;
|
||||
fields: FieldDefinition[];
|
||||
relations?: RelationDefinition[];
|
||||
}
|
||||
|
||||
export class DynamicModelFactory {
|
||||
/**
|
||||
* Get relation name from lookup field API name
|
||||
* Converts "ownerId" -> "owner", "customFieldId" -> "customfield"
|
||||
*/
|
||||
static getRelationName(lookupFieldApiName: string): string {
|
||||
return lookupFieldApiName.replace(/Id$/, '').toLowerCase();
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a dynamic model class from object metadata
|
||||
* @param meta Object metadata
|
||||
* @param getModel Function to retrieve model classes from registry
|
||||
*/
|
||||
static createModel(
|
||||
meta: ObjectMetadata,
|
||||
getModel?: (apiName: string) => ModelClass<any>,
|
||||
): ModelClass<any> {
|
||||
const { tableName, fields, apiName, relations = [] } = meta;
|
||||
|
||||
// Build JSON schema properties
|
||||
const properties: Record<string, any> = {
|
||||
id: { type: 'string' },
|
||||
tenantId: { type: 'string' },
|
||||
ownerId: { type: 'string' },
|
||||
name: { type: 'string' },
|
||||
created_at: { type: 'string', format: 'date-time' },
|
||||
updated_at: { type: 'string', format: 'date-time' },
|
||||
};
|
||||
|
||||
// Don't require id or tenantId - they'll be set automatically
|
||||
const required: string[] = [];
|
||||
|
||||
// Add custom fields
|
||||
for (const field of fields) {
|
||||
properties[field.apiName] = this.fieldToJsonSchema(field);
|
||||
|
||||
// Only mark as required if explicitly required AND not a system field
|
||||
const systemFields = ['id', 'tenantId', 'ownerId', 'name', 'created_at', 'updated_at'];
|
||||
if (field.isRequired && !systemFields.includes(field.apiName)) {
|
||||
required.push(field.apiName);
|
||||
}
|
||||
}
|
||||
|
||||
// Build relation mappings from lookup fields
|
||||
const lookupFields = fields.filter(f => f.type === 'LOOKUP' && f.referenceObject);
|
||||
|
||||
// Store lookup fields metadata for later use
|
||||
const lookupFieldsInfo = lookupFields.map(f => ({
|
||||
apiName: f.apiName,
|
||||
relationName: DynamicModelFactory.getRelationName(f.apiName),
|
||||
referenceObject: f.referenceObject,
|
||||
targetTable: this.getTableName(f.referenceObject),
|
||||
}));
|
||||
|
||||
// Create the dynamic model class extending BaseModel
|
||||
class DynamicModel extends BaseModel {
|
||||
static tableName = tableName;
|
||||
|
||||
static objectApiName = apiName;
|
||||
|
||||
static lookupFields = lookupFieldsInfo;
|
||||
|
||||
static get relationMappings(): RelationMappings {
|
||||
const mappings: RelationMappings = {};
|
||||
|
||||
// Build relation mappings from lookup fields
|
||||
for (const lookupInfo of lookupFieldsInfo) {
|
||||
// Use getModel function if provided, otherwise use string reference
|
||||
let modelClass: any = lookupInfo.referenceObject;
|
||||
|
||||
if (getModel) {
|
||||
const resolvedModel = getModel(lookupInfo.referenceObject);
|
||||
// Only use resolved model if it exists, otherwise skip this relation
|
||||
// It will be resolved later when the model is registered
|
||||
if (resolvedModel) {
|
||||
modelClass = resolvedModel;
|
||||
} else {
|
||||
// Skip this relation if model not found yet
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
mappings[lookupInfo.relationName] = {
|
||||
relation: Model.BelongsToOneRelation,
|
||||
modelClass,
|
||||
join: {
|
||||
from: `${tableName}.${lookupInfo.apiName}`,
|
||||
to: `${lookupInfo.targetTable}.id`,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Add additional relation mappings (e.g., hasMany)
|
||||
for (const relation of relations) {
|
||||
if (mappings[relation.name]) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let modelClass: any = relation.targetObjectApiName;
|
||||
if (getModel) {
|
||||
const resolvedModel = getModel(relation.targetObjectApiName);
|
||||
if (resolvedModel) {
|
||||
modelClass = resolvedModel;
|
||||
} else {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
const targetTable = DynamicModelFactory.getTableName(relation.targetObjectApiName);
|
||||
|
||||
if (relation.type === 'belongsTo') {
|
||||
mappings[relation.name] = {
|
||||
relation: Model.BelongsToOneRelation,
|
||||
modelClass,
|
||||
join: {
|
||||
from: `${tableName}.${relation.fromColumn}`,
|
||||
to: `${targetTable}.${relation.toColumn}`,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
if (relation.type === 'hasMany') {
|
||||
mappings[relation.name] = {
|
||||
relation: Model.HasManyRelation,
|
||||
modelClass,
|
||||
join: {
|
||||
from: `${tableName}.${relation.fromColumn}`,
|
||||
to: `${targetTable}.${relation.toColumn}`,
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return mappings;
|
||||
}
|
||||
|
||||
static get jsonSchema() {
|
||||
return {
|
||||
type: 'object',
|
||||
required,
|
||||
properties,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return DynamicModel as any;
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert a field definition to JSON schema property
|
||||
*/
|
||||
private static fieldToJsonSchema(field: FieldDefinition): Record<string, any> {
|
||||
const baseSchema = () => {
|
||||
switch (field.type.toUpperCase()) {
|
||||
case 'TEXT':
|
||||
case 'STRING':
|
||||
case 'EMAIL':
|
||||
case 'URL':
|
||||
case 'PHONE':
|
||||
case 'PICKLIST':
|
||||
case 'MULTI_PICKLIST':
|
||||
return {
|
||||
type: 'string',
|
||||
...(field.isUnique && { uniqueItems: true }),
|
||||
};
|
||||
|
||||
case 'LONG_TEXT':
|
||||
return { type: 'string' };
|
||||
|
||||
case 'NUMBER':
|
||||
case 'DECIMAL':
|
||||
case 'CURRENCY':
|
||||
case 'PERCENT':
|
||||
return {
|
||||
type: 'number',
|
||||
...(field.isUnique && { uniqueItems: true }),
|
||||
};
|
||||
|
||||
case 'INTEGER':
|
||||
return {
|
||||
type: 'integer',
|
||||
...(field.isUnique && { uniqueItems: true }),
|
||||
};
|
||||
|
||||
case 'BOOLEAN':
|
||||
return { type: 'boolean', default: false };
|
||||
|
||||
case 'DATE':
|
||||
return { type: 'string', format: 'date' };
|
||||
|
||||
case 'DATE_TIME':
|
||||
return { type: 'string', format: 'date-time' };
|
||||
|
||||
case 'LOOKUP':
|
||||
case 'BELONGS_TO':
|
||||
return { type: 'string' };
|
||||
|
||||
default:
|
||||
return { type: 'string' };
|
||||
}
|
||||
};
|
||||
|
||||
const schema = baseSchema();
|
||||
|
||||
// Allow null for non-required fields so optional strings/numbers don't fail validation
|
||||
if (!field.isRequired) {
|
||||
return {
|
||||
anyOf: [schema, { type: 'null' }],
|
||||
};
|
||||
}
|
||||
|
||||
return schema;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get table name from object API name
|
||||
*/
|
||||
private static getTableName(objectApiName: string): string {
|
||||
// Convert PascalCase/camelCase to snake_case and pluralize
|
||||
const snakeCase = objectApiName
|
||||
.replace(/([A-Z])/g, '_$1')
|
||||
.toLowerCase()
|
||||
.replace(/^_/, '');
|
||||
if (snakeCase.endsWith('y')) {
|
||||
return `${snakeCase.slice(0, -1)}ies`;
|
||||
}
|
||||
return snakeCase.endsWith('s') ? snakeCase : `${snakeCase}s`;
|
||||
}
|
||||
}
|
||||
73
backend/src/object/models/model.registry.ts
Normal file
73
backend/src/object/models/model.registry.ts
Normal file
@@ -0,0 +1,73 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import { ModelClass } from 'objection';
|
||||
import { BaseModel } from './base.model';
|
||||
import { DynamicModelFactory, ObjectMetadata } from './dynamic-model.factory';
|
||||
|
||||
/**
|
||||
* Registry to store and retrieve dynamic models
|
||||
* One registry per tenant
|
||||
*/
|
||||
@Injectable()
|
||||
export class ModelRegistry {
|
||||
private registry = new Map<string, ModelClass<BaseModel>>();
|
||||
|
||||
/**
|
||||
* Register a model in the registry
|
||||
*/
|
||||
registerModel(apiName: string, modelClass: ModelClass<BaseModel>): void {
|
||||
this.registry.set(apiName, modelClass);
|
||||
const lowerKey = apiName.toLowerCase();
|
||||
if (lowerKey !== apiName && !this.registry.has(lowerKey)) {
|
||||
this.registry.set(lowerKey, modelClass);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a model from the registry
|
||||
*/
|
||||
getModel(apiName: string): ModelClass<BaseModel> {
|
||||
const model = this.registry.get(apiName) || this.registry.get(apiName.toLowerCase());
|
||||
if (!model) {
|
||||
throw new Error(`Model for ${apiName} not found in registry`);
|
||||
}
|
||||
return model;
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a model exists in the registry
|
||||
*/
|
||||
hasModel(apiName: string): boolean {
|
||||
return this.registry.has(apiName) || this.registry.has(apiName.toLowerCase());
|
||||
}
|
||||
|
||||
/**
|
||||
* Create and register a model from metadata
|
||||
*/
|
||||
createAndRegisterModel(
|
||||
metadata: ObjectMetadata,
|
||||
): ModelClass<BaseModel> {
|
||||
// Create model with a getModel function that resolves from this registry
|
||||
// Returns undefined if model not found (for models not yet registered)
|
||||
const model = DynamicModelFactory.createModel(
|
||||
metadata,
|
||||
(apiName: string) =>
|
||||
this.registry.get(apiName) || this.registry.get(apiName.toLowerCase()),
|
||||
);
|
||||
this.registerModel(metadata.apiName, model);
|
||||
return model;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all registered model names
|
||||
*/
|
||||
getAllModelNames(): string[] {
|
||||
return Array.from(this.registry.keys());
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear the registry (useful for testing)
|
||||
*/
|
||||
clear(): void {
|
||||
this.registry.clear();
|
||||
}
|
||||
}
|
||||
203
backend/src/object/models/model.service.ts
Normal file
203
backend/src/object/models/model.service.ts
Normal file
@@ -0,0 +1,203 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
import { Knex } from 'knex';
|
||||
import { ModelClass } from 'objection';
|
||||
import { BaseModel } from './base.model';
|
||||
import { ModelRegistry } from './model.registry';
|
||||
import { ObjectMetadata } from './dynamic-model.factory';
|
||||
import { TenantDatabaseService } from '../../tenant/tenant-database.service';
|
||||
import { UserModel, RoleModel, PermissionModel } from './system-models';
|
||||
|
||||
/**
|
||||
* Service to manage dynamic models for a specific tenant
|
||||
*/
|
||||
@Injectable()
|
||||
export class ModelService {
|
||||
private readonly logger = new Logger(ModelService.name);
|
||||
private tenantRegistries = new Map<string, ModelRegistry>();
|
||||
|
||||
constructor(private tenantDbService: TenantDatabaseService) {}
|
||||
|
||||
/**
|
||||
* Get or create a registry for a tenant
|
||||
*/
|
||||
getTenantRegistry(tenantId: string): ModelRegistry {
|
||||
if (!this.tenantRegistries.has(tenantId)) {
|
||||
const registry = new ModelRegistry();
|
||||
|
||||
// Register system models that are defined as static Objection models
|
||||
this.registerSystemModels(registry);
|
||||
|
||||
this.tenantRegistries.set(tenantId, registry);
|
||||
}
|
||||
return this.tenantRegistries.get(tenantId)!;
|
||||
}
|
||||
|
||||
/**
|
||||
* Register static system models in the registry
|
||||
* Uses simplified models without complex relationMappings to avoid modelPath issues
|
||||
*/
|
||||
private registerSystemModels(registry: ModelRegistry): void {
|
||||
// Register system models by their API name (used in referenceObject fields)
|
||||
// These are simplified versions without relationMappings to avoid dependency issues
|
||||
registry.registerModel('User', UserModel as any);
|
||||
registry.registerModel('Role', RoleModel as any);
|
||||
registry.registerModel('Permission', PermissionModel as any);
|
||||
|
||||
this.logger.debug('Registered system models: User, Role, Permission');
|
||||
}
|
||||
|
||||
/**
|
||||
* Create and register a model for a tenant
|
||||
*/
|
||||
async createModelForObject(
|
||||
tenantId: string,
|
||||
objectMetadata: ObjectMetadata,
|
||||
): Promise<ModelClass<BaseModel>> {
|
||||
const registry = this.getTenantRegistry(tenantId);
|
||||
const model = registry.createAndRegisterModel(objectMetadata);
|
||||
|
||||
this.logger.log(
|
||||
`Registered model for ${objectMetadata.apiName} in tenant ${tenantId}`,
|
||||
);
|
||||
|
||||
return model;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a model for a tenant and object
|
||||
*/
|
||||
getModel(tenantId: string, objectApiName: string): ModelClass<BaseModel> {
|
||||
const registry = this.getTenantRegistry(tenantId);
|
||||
return registry.getModel(objectApiName);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a bound model (with knex connection) for a tenant and object
|
||||
*/
|
||||
async getBoundModel(
|
||||
tenantId: string,
|
||||
objectApiName: string,
|
||||
): Promise<ModelClass<BaseModel>> {
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
const model = this.getModel(tenantId, objectApiName);
|
||||
|
||||
// Bind knex to the model and also to all models in the registry
|
||||
// This ensures system models also have knex bound when they're used in relations
|
||||
const registry = this.getTenantRegistry(tenantId);
|
||||
const allModels = registry.getAllModelNames();
|
||||
|
||||
// Bind knex to all models to ensure relations work
|
||||
for (const modelName of allModels) {
|
||||
try {
|
||||
const m = registry.getModel(modelName);
|
||||
if (m && !m.knex()) {
|
||||
m.knex(knex);
|
||||
}
|
||||
} catch (error) {
|
||||
// Ignore errors for models that don't need binding
|
||||
}
|
||||
}
|
||||
|
||||
return model.bindKnex(knex);
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if a model exists for a tenant
|
||||
*/
|
||||
hasModel(tenantId: string, objectApiName: string): boolean {
|
||||
const registry = this.getTenantRegistry(tenantId);
|
||||
return registry.hasModel(objectApiName);
|
||||
}
|
||||
|
||||
/**
|
||||
* Get all model names for a tenant
|
||||
*/
|
||||
getAllModelNames(tenantId: string): string[] {
|
||||
const registry = this.getTenantRegistry(tenantId);
|
||||
return registry.getAllModelNames();
|
||||
}
|
||||
|
||||
/**
|
||||
* Ensure a model is registered with all its dependencies.
|
||||
* This method handles recursive model creation for related objects.
|
||||
*
|
||||
* @param tenantId - The tenant ID
|
||||
* @param objectApiName - The object API name to ensure registration for
|
||||
* @param fetchMetadata - Callback function to fetch object metadata (provided by ObjectService)
|
||||
* @param visited - Set to track visited models and prevent infinite loops
|
||||
*/
|
||||
async ensureModelWithDependencies(
|
||||
tenantId: string,
|
||||
objectApiName: string,
|
||||
fetchMetadata: (apiName: string) => Promise<ObjectMetadata>,
|
||||
visited: Set<string> = new Set(),
|
||||
): Promise<void> {
|
||||
// Prevent infinite recursion
|
||||
if (visited.has(objectApiName)) {
|
||||
return;
|
||||
}
|
||||
visited.add(objectApiName);
|
||||
|
||||
// Check if model already exists
|
||||
if (this.hasModel(tenantId, objectApiName)) {
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
// Fetch the object metadata
|
||||
const objectMetadata = await fetchMetadata(objectApiName);
|
||||
|
||||
// Extract lookup fields to find dependencies
|
||||
const lookupFields = objectMetadata.fields.filter(
|
||||
f => f.type === 'LOOKUP' && f.referenceObject
|
||||
);
|
||||
|
||||
// Recursively ensure all dependent models are registered first
|
||||
for (const field of lookupFields) {
|
||||
if (field.referenceObject) {
|
||||
try {
|
||||
await this.ensureModelWithDependencies(
|
||||
tenantId,
|
||||
field.referenceObject,
|
||||
fetchMetadata,
|
||||
visited,
|
||||
);
|
||||
} catch (error) {
|
||||
// If related object doesn't exist (e.g., system tables), skip it
|
||||
this.logger.debug(
|
||||
`Skipping registration of related model ${field.referenceObject}: ${error.message}`
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (objectMetadata.relations) {
|
||||
for (const relation of objectMetadata.relations) {
|
||||
if (relation.targetObjectApiName) {
|
||||
try {
|
||||
await this.ensureModelWithDependencies(
|
||||
tenantId,
|
||||
relation.targetObjectApiName,
|
||||
fetchMetadata,
|
||||
visited,
|
||||
);
|
||||
} catch (error) {
|
||||
this.logger.debug(
|
||||
`Skipping registration of related model ${relation.targetObjectApiName}: ${error.message}`
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Now create and register this model (all dependencies are ready)
|
||||
await this.createModelForObject(tenantId, objectMetadata);
|
||||
this.logger.log(`Registered model for ${objectApiName} in tenant ${tenantId}`);
|
||||
} catch (error) {
|
||||
this.logger.warn(
|
||||
`Failed to ensure model for ${objectApiName}: ${error.message}`
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
85
backend/src/object/models/system-models.ts
Normal file
85
backend/src/object/models/system-models.ts
Normal file
@@ -0,0 +1,85 @@
|
||||
import { Model } from 'objection';
|
||||
|
||||
/**
|
||||
* Simplified User model for use in dynamic object relations
|
||||
* This version doesn't include complex relationMappings to avoid modelPath issues
|
||||
*/
|
||||
export class UserModel extends Model {
|
||||
static tableName = 'users';
|
||||
static objectApiName = 'User';
|
||||
|
||||
id!: string;
|
||||
email!: string;
|
||||
firstName?: string;
|
||||
lastName?: string;
|
||||
name?: string;
|
||||
isActive!: boolean;
|
||||
createdAt!: Date;
|
||||
updatedAt!: Date;
|
||||
|
||||
static get jsonSchema() {
|
||||
return {
|
||||
type: 'object',
|
||||
required: ['email'],
|
||||
properties: {
|
||||
id: { type: 'string' },
|
||||
email: { type: 'string', format: 'email' },
|
||||
firstName: { type: 'string' },
|
||||
lastName: { type: 'string' },
|
||||
name: { type: 'string' },
|
||||
isActive: { type: 'boolean' },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// No relationMappings to avoid modelPath resolution issues
|
||||
// These simplified models are only used for lookup relations from dynamic models
|
||||
}
|
||||
|
||||
/**
|
||||
* Simplified Role model for use in dynamic object relations
|
||||
*/
|
||||
export class RoleModel extends Model {
|
||||
static tableName = 'roles';
|
||||
static objectApiName = 'Role';
|
||||
|
||||
id!: string;
|
||||
name!: string;
|
||||
description?: string;
|
||||
|
||||
static get jsonSchema() {
|
||||
return {
|
||||
type: 'object',
|
||||
required: ['name'],
|
||||
properties: {
|
||||
id: { type: 'string' },
|
||||
name: { type: 'string' },
|
||||
description: { type: 'string' },
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Simplified Permission model for use in dynamic object relations
|
||||
*/
|
||||
export class PermissionModel extends Model {
|
||||
static tableName = 'permissions';
|
||||
static objectApiName = 'Permission';
|
||||
|
||||
id!: string;
|
||||
name!: string;
|
||||
description?: string;
|
||||
|
||||
static get jsonSchema() {
|
||||
return {
|
||||
type: 'object',
|
||||
required: ['name'],
|
||||
properties: {
|
||||
id: { type: 'string' },
|
||||
name: { type: 'string' },
|
||||
description: { type: 'string' },
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -5,11 +5,23 @@ import { SetupObjectController } from './setup-object.controller';
|
||||
import { SchemaManagementService } from './schema-management.service';
|
||||
import { FieldMapperService } from './field-mapper.service';
|
||||
import { TenantModule } from '../tenant/tenant.module';
|
||||
import { MigrationModule } from '../migration/migration.module';
|
||||
import { RbacModule } from '../rbac/rbac.module';
|
||||
import { ModelRegistry } from './models/model.registry';
|
||||
import { ModelService } from './models/model.service';
|
||||
import { MeilisearchModule } from '../search/meilisearch.module';
|
||||
import { KnowledgeModule } from '../knowledge/knowledge.module';
|
||||
|
||||
@Module({
|
||||
imports: [TenantModule],
|
||||
providers: [ObjectService, SchemaManagementService, FieldMapperService],
|
||||
imports: [TenantModule, MigrationModule, RbacModule, MeilisearchModule, KnowledgeModule],
|
||||
providers: [
|
||||
ObjectService,
|
||||
SchemaManagementService,
|
||||
FieldMapperService,
|
||||
ModelRegistry,
|
||||
ModelService,
|
||||
],
|
||||
controllers: [RuntimeObjectController, SetupObjectController],
|
||||
exports: [ObjectService, SchemaManagementService, FieldMapperService],
|
||||
exports: [ObjectService, SchemaManagementService, FieldMapperService, ModelService],
|
||||
})
|
||||
export class ObjectModule {}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -95,4 +95,49 @@ export class RuntimeObjectController {
|
||||
user.userId,
|
||||
);
|
||||
}
|
||||
|
||||
@Post(':objectApiName/records/bulk-delete')
|
||||
async deleteRecords(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Body() body: { recordIds?: string[]; ids?: string[] },
|
||||
@CurrentUser() user: any,
|
||||
) {
|
||||
const recordIds: string[] = body?.recordIds || body?.ids || [];
|
||||
return this.objectService.deleteRecords(
|
||||
tenantId,
|
||||
objectApiName,
|
||||
recordIds,
|
||||
user.userId,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Direct filter-based search — used when applying a saved list view.
|
||||
* Bypasses the AI planning step; accepts pre-resolved structured filters.
|
||||
*/
|
||||
@Post(':objectApiName/records/search')
|
||||
async searchRecords(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@CurrentUser() user: any,
|
||||
@Body() body: {
|
||||
filters?: Array<{ field: string; operator: string; value?: any; values?: any[]; from?: string; to?: string }>;
|
||||
sort?: { field: string; direction: 'asc' | 'desc' } | null;
|
||||
page?: number;
|
||||
pageSize?: number;
|
||||
},
|
||||
) {
|
||||
const page = Number.isFinite(Number(body?.page)) ? Number(body.page) : 1;
|
||||
const pageSize = Number.isFinite(Number(body?.pageSize)) ? Number(body.pageSize) : 25;
|
||||
|
||||
return this.objectService.searchRecordsWithFilters(
|
||||
tenantId,
|
||||
objectApiName,
|
||||
user.userId,
|
||||
body?.filters || [],
|
||||
{ page, pageSize },
|
||||
body?.sort || undefined,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15,7 +15,11 @@ export class SchemaManagementService {
|
||||
objectDefinition: ObjectDefinition,
|
||||
fields: FieldDefinition[],
|
||||
) {
|
||||
const tableName = this.getTableName(objectDefinition.apiName);
|
||||
const tableName = this.getTableName(
|
||||
objectDefinition.apiName,
|
||||
objectDefinition.label,
|
||||
objectDefinition.pluralLabel,
|
||||
);
|
||||
|
||||
// Check if table already exists
|
||||
const exists = await knex.schema.hasTable(tableName);
|
||||
@@ -44,8 +48,10 @@ export class SchemaManagementService {
|
||||
knex: Knex,
|
||||
objectApiName: string,
|
||||
field: FieldDefinition,
|
||||
objectLabel?: string,
|
||||
pluralLabel?: string,
|
||||
) {
|
||||
const tableName = this.getTableName(objectApiName);
|
||||
const tableName = this.getTableName(objectApiName, objectLabel, pluralLabel);
|
||||
|
||||
await knex.schema.alterTable(tableName, (table) => {
|
||||
this.addFieldColumn(table, field);
|
||||
@@ -61,8 +67,10 @@ export class SchemaManagementService {
|
||||
knex: Knex,
|
||||
objectApiName: string,
|
||||
fieldApiName: string,
|
||||
objectLabel?: string,
|
||||
pluralLabel?: string,
|
||||
) {
|
||||
const tableName = this.getTableName(objectApiName);
|
||||
const tableName = this.getTableName(objectApiName, objectLabel, pluralLabel);
|
||||
|
||||
await knex.schema.alterTable(tableName, (table) => {
|
||||
table.dropColumn(fieldApiName);
|
||||
@@ -71,11 +79,44 @@ export class SchemaManagementService {
|
||||
this.logger.log(`Removed field ${fieldApiName} from table ${tableName}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Alter a field in an existing object table
|
||||
* Handles safe updates like changing NOT NULL or constraints
|
||||
* Warns about potentially destructive operations
|
||||
*/
|
||||
async alterFieldInTable(
|
||||
knex: Knex,
|
||||
objectApiName: string,
|
||||
fieldApiName: string,
|
||||
field: FieldDefinition,
|
||||
objectLabel?: string,
|
||||
pluralLabel?: string,
|
||||
options?: {
|
||||
skipTypeChange?: boolean; // Skip if type change would lose data
|
||||
},
|
||||
) {
|
||||
const tableName = this.getTableName(objectApiName, objectLabel, pluralLabel);
|
||||
const skipTypeChange = options?.skipTypeChange ?? true;
|
||||
|
||||
await knex.schema.alterTable(tableName, (table) => {
|
||||
// Drop the existing column and recreate with new definition
|
||||
// Note: This approach works for metadata changes, but type changes may need data migration
|
||||
table.dropColumn(fieldApiName);
|
||||
});
|
||||
|
||||
// Recreate the column with new definition
|
||||
await knex.schema.alterTable(tableName, (table) => {
|
||||
this.addFieldColumn(table, field);
|
||||
});
|
||||
|
||||
this.logger.log(`Altered field ${fieldApiName} in table ${tableName}`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop an object table
|
||||
*/
|
||||
async dropObjectTable(knex: Knex, objectApiName: string) {
|
||||
const tableName = this.getTableName(objectApiName);
|
||||
async dropObjectTable(knex: Knex, objectApiName: string, objectLabel?: string, pluralLabel?: string) {
|
||||
const tableName = this.getTableName(objectApiName, objectLabel, pluralLabel);
|
||||
|
||||
await knex.schema.dropTableIfExists(tableName);
|
||||
|
||||
@@ -94,15 +135,30 @@ export class SchemaManagementService {
|
||||
let column: Knex.ColumnBuilder;
|
||||
|
||||
switch (field.type) {
|
||||
// Text types
|
||||
case 'String':
|
||||
case 'TEXT':
|
||||
case 'EMAIL':
|
||||
case 'PHONE':
|
||||
case 'URL':
|
||||
column = table.string(columnName, field.length || 255);
|
||||
break;
|
||||
|
||||
case 'Text':
|
||||
case 'LONG_TEXT':
|
||||
column = table.text(columnName);
|
||||
break;
|
||||
|
||||
case 'PICKLIST':
|
||||
case 'MULTI_PICKLIST':
|
||||
column = table.string(columnName, 255);
|
||||
break;
|
||||
|
||||
// Numeric types
|
||||
case 'Number':
|
||||
case 'NUMBER':
|
||||
case 'CURRENCY':
|
||||
case 'PERCENT':
|
||||
if (field.scale && field.scale > 0) {
|
||||
column = table.decimal(
|
||||
columnName,
|
||||
@@ -115,18 +171,28 @@ export class SchemaManagementService {
|
||||
break;
|
||||
|
||||
case 'Boolean':
|
||||
case 'BOOLEAN':
|
||||
column = table.boolean(columnName).defaultTo(false);
|
||||
break;
|
||||
|
||||
// Date types
|
||||
case 'Date':
|
||||
case 'DATE':
|
||||
column = table.date(columnName);
|
||||
break;
|
||||
|
||||
case 'DateTime':
|
||||
case 'DATE_TIME':
|
||||
column = table.datetime(columnName);
|
||||
break;
|
||||
|
||||
case 'TIME':
|
||||
column = table.time(columnName);
|
||||
break;
|
||||
|
||||
// Relationship types
|
||||
case 'Reference':
|
||||
case 'LOOKUP':
|
||||
column = table.uuid(columnName);
|
||||
if (field.referenceObject) {
|
||||
const refTableName = this.getTableName(field.referenceObject);
|
||||
@@ -134,19 +200,30 @@ export class SchemaManagementService {
|
||||
}
|
||||
break;
|
||||
|
||||
// Email (legacy)
|
||||
case 'Email':
|
||||
column = table.string(columnName, 255);
|
||||
break;
|
||||
|
||||
// Phone (legacy)
|
||||
case 'Phone':
|
||||
column = table.string(columnName, 50);
|
||||
break;
|
||||
|
||||
// Url (legacy)
|
||||
case 'Url':
|
||||
column = table.string(columnName, 255);
|
||||
break;
|
||||
|
||||
// File types
|
||||
case 'FILE':
|
||||
case 'IMAGE':
|
||||
column = table.text(columnName); // Store file path or URL
|
||||
break;
|
||||
|
||||
// JSON
|
||||
case 'Json':
|
||||
case 'JSON':
|
||||
column = table.json(columnName);
|
||||
break;
|
||||
|
||||
@@ -174,16 +251,35 @@ export class SchemaManagementService {
|
||||
/**
|
||||
* Convert object API name to table name (convert to snake_case, pluralize)
|
||||
*/
|
||||
private getTableName(apiName: string): string {
|
||||
// Convert PascalCase to snake_case
|
||||
const snakeCase = apiName
|
||||
.replace(/([A-Z])/g, '_$1')
|
||||
private getTableName(apiName: string, objectLabel?: string, pluralLabel?: string): string {
|
||||
const toSnakePlural = (source: string): string => {
|
||||
const cleaned = source.replace(/[\s-]+/g, '_');
|
||||
const snake = cleaned
|
||||
.replace(/([a-z0-9])([A-Z])/g, '$1_$2')
|
||||
.replace(/__+/g, '_')
|
||||
.toLowerCase()
|
||||
.replace(/^_/, '');
|
||||
|
||||
// Simple pluralization (append 's' if not already plural)
|
||||
// In production, use a proper pluralization library
|
||||
return snakeCase.endsWith('s') ? snakeCase : `${snakeCase}s`;
|
||||
if (snake.endsWith('y')) return `${snake.slice(0, -1)}ies`;
|
||||
if (snake.endsWith('s')) return snake;
|
||||
return `${snake}s`;
|
||||
};
|
||||
|
||||
const fromApi = toSnakePlural(apiName);
|
||||
const fromLabel = objectLabel ? toSnakePlural(objectLabel) : null;
|
||||
const fromPlural = pluralLabel ? toSnakePlural(pluralLabel) : null;
|
||||
|
||||
if (fromLabel && fromLabel.includes('_') && !fromApi.includes('_')) {
|
||||
return fromLabel;
|
||||
}
|
||||
if (fromPlural && fromPlural.includes('_') && !fromApi.includes('_')) {
|
||||
return fromPlural;
|
||||
}
|
||||
|
||||
if (fromLabel && fromLabel !== fromApi) return fromLabel;
|
||||
if (fromPlural && fromPlural !== fromApi) return fromPlural;
|
||||
|
||||
return fromApi;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -2,6 +2,9 @@ import {
|
||||
Controller,
|
||||
Get,
|
||||
Post,
|
||||
Patch,
|
||||
Put,
|
||||
Delete,
|
||||
Param,
|
||||
Body,
|
||||
UseGuards,
|
||||
@@ -10,6 +13,7 @@ import { ObjectService } from './object.service';
|
||||
import { FieldMapperService } from './field-mapper.service';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { TenantId } from '../tenant/tenant.decorator';
|
||||
import { TenantDatabaseService } from '../tenant/tenant-database.service';
|
||||
|
||||
@Controller('setup/objects')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
@@ -17,6 +21,7 @@ export class SetupObjectController {
|
||||
constructor(
|
||||
private objectService: ObjectService,
|
||||
private fieldMapperService: FieldMapperService,
|
||||
private tenantDbService: TenantDatabaseService,
|
||||
) {}
|
||||
|
||||
@Get()
|
||||
@@ -29,7 +34,8 @@ export class SetupObjectController {
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
) {
|
||||
return this.objectService.getObjectDefinition(tenantId, objectApiName);
|
||||
const objectDef = await this.objectService.getObjectDefinition(tenantId, objectApiName);
|
||||
return this.fieldMapperService.mapObjectDefinitionToDTO(objectDef);
|
||||
}
|
||||
|
||||
@Get(':objectApiName/ui-config')
|
||||
@@ -58,10 +64,93 @@ export class SetupObjectController {
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Body() data: any,
|
||||
) {
|
||||
return this.objectService.createFieldDefinition(
|
||||
const field = await this.objectService.createFieldDefinition(
|
||||
tenantId,
|
||||
objectApiName,
|
||||
data,
|
||||
);
|
||||
// Map the created field to frontend format
|
||||
return this.fieldMapperService.mapFieldToDTO(field);
|
||||
}
|
||||
|
||||
@Put(':objectApiName/fields/:fieldApiName')
|
||||
async updateFieldDefinition(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Param('fieldApiName') fieldApiName: string,
|
||||
@Body() data: any,
|
||||
) {
|
||||
const field = await this.objectService.updateFieldDefinition(
|
||||
tenantId,
|
||||
objectApiName,
|
||||
fieldApiName,
|
||||
data,
|
||||
);
|
||||
return this.fieldMapperService.mapFieldToDTO(field);
|
||||
}
|
||||
|
||||
@Delete(':objectApiName/fields/:fieldApiName')
|
||||
async deleteFieldDefinition(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Param('fieldApiName') fieldApiName: string,
|
||||
) {
|
||||
return this.objectService.deleteFieldDefinition(
|
||||
tenantId,
|
||||
objectApiName,
|
||||
fieldApiName,
|
||||
);
|
||||
}
|
||||
|
||||
@Patch(':objectApiName')
|
||||
async updateObjectDefinition(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Body() data: any,
|
||||
) {
|
||||
return this.objectService.updateObjectDefinition(tenantId, objectApiName, data);
|
||||
}
|
||||
|
||||
@Get(':objectId/field-permissions')
|
||||
async getFieldPermissions(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectId') objectId: string,
|
||||
) {
|
||||
return this.objectService.getFieldPermissions(tenantId, objectId);
|
||||
}
|
||||
|
||||
@Put(':objectId/field-permissions')
|
||||
async updateFieldPermission(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectId') objectId: string,
|
||||
@Body() data: { roleId: string; fieldDefinitionId: string; canRead: boolean; canEdit: boolean },
|
||||
) {
|
||||
return this.objectService.updateFieldPermission(tenantId, data.roleId, data.fieldDefinitionId, data.canRead, data.canEdit);
|
||||
}
|
||||
|
||||
@Get(':objectApiName/permissions/:roleId')
|
||||
async getObjectPermissions(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Param('roleId') roleId: string,
|
||||
) {
|
||||
return this.objectService.getObjectPermissions(tenantId, objectApiName, roleId);
|
||||
}
|
||||
|
||||
@Put(':objectApiName/permissions')
|
||||
async updateObjectPermissions(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Body() data: {
|
||||
roleId: string;
|
||||
canCreate: boolean;
|
||||
canRead: boolean;
|
||||
canEdit: boolean;
|
||||
canDelete: boolean;
|
||||
canViewAll: boolean;
|
||||
canModifyAll: boolean;
|
||||
},
|
||||
) {
|
||||
return this.objectService.updateObjectPermissions(tenantId, objectApiName, data);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
import { IsString, IsUUID, IsBoolean, IsOptional, IsObject } from 'class-validator';
|
||||
import { IsString, IsUUID, IsBoolean, IsOptional, IsObject, IsIn } from 'class-validator';
|
||||
|
||||
export type PageLayoutType = 'detail' | 'list';
|
||||
|
||||
export class CreatePageLayoutDto {
|
||||
@IsString()
|
||||
@@ -7,19 +9,27 @@ export class CreatePageLayoutDto {
|
||||
@IsUUID()
|
||||
objectId: string;
|
||||
|
||||
@IsIn(['detail', 'list'])
|
||||
@IsOptional()
|
||||
layoutType?: PageLayoutType = 'detail';
|
||||
|
||||
@IsBoolean()
|
||||
@IsOptional()
|
||||
isDefault?: boolean;
|
||||
|
||||
@IsObject()
|
||||
layoutConfig: {
|
||||
// For detail layouts: grid-based field positions
|
||||
fields: Array<{
|
||||
fieldId: string;
|
||||
x: number;
|
||||
y: number;
|
||||
w: number;
|
||||
h: number;
|
||||
x?: number;
|
||||
y?: number;
|
||||
w?: number;
|
||||
h?: number;
|
||||
// For list layouts: field order (optional, defaults to array index)
|
||||
order?: number;
|
||||
}>;
|
||||
relatedLists?: string[];
|
||||
};
|
||||
|
||||
@IsString()
|
||||
@@ -41,11 +51,13 @@ export class UpdatePageLayoutDto {
|
||||
layoutConfig?: {
|
||||
fields: Array<{
|
||||
fieldId: string;
|
||||
x: number;
|
||||
y: number;
|
||||
w: number;
|
||||
h: number;
|
||||
x?: number;
|
||||
y?: number;
|
||||
w?: number;
|
||||
h?: number;
|
||||
order?: number;
|
||||
}>;
|
||||
relatedLists?: string[];
|
||||
};
|
||||
|
||||
@IsString()
|
||||
|
||||
@@ -10,7 +10,7 @@ import {
|
||||
Query,
|
||||
} from '@nestjs/common';
|
||||
import { PageLayoutService } from './page-layout.service';
|
||||
import { CreatePageLayoutDto, UpdatePageLayoutDto } from './dto/page-layout.dto';
|
||||
import { CreatePageLayoutDto, UpdatePageLayoutDto, PageLayoutType } from './dto/page-layout.dto';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { TenantId } from '../tenant/tenant.decorator';
|
||||
|
||||
@@ -25,13 +25,21 @@ export class PageLayoutController {
|
||||
}
|
||||
|
||||
@Get()
|
||||
findAll(@TenantId() tenantId: string, @Query('objectId') objectId?: string) {
|
||||
return this.pageLayoutService.findAll(tenantId, objectId);
|
||||
findAll(
|
||||
@TenantId() tenantId: string,
|
||||
@Query('objectId') objectId?: string,
|
||||
@Query('layoutType') layoutType?: PageLayoutType,
|
||||
) {
|
||||
return this.pageLayoutService.findAll(tenantId, objectId, layoutType);
|
||||
}
|
||||
|
||||
@Get('default/:objectId')
|
||||
findDefaultByObject(@TenantId() tenantId: string, @Param('objectId') objectId: string) {
|
||||
return this.pageLayoutService.findDefaultByObject(tenantId, objectId);
|
||||
findDefaultByObject(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectId') objectId: string,
|
||||
@Query('layoutType') layoutType?: PageLayoutType,
|
||||
) {
|
||||
return this.pageLayoutService.findDefaultByObject(tenantId, objectId, layoutType || 'detail');
|
||||
}
|
||||
|
||||
@Get(':id')
|
||||
|
||||
@@ -1,24 +1,26 @@
|
||||
import { Injectable, NotFoundException } from '@nestjs/common';
|
||||
import { TenantDatabaseService } from '../tenant/tenant-database.service';
|
||||
import { CreatePageLayoutDto, UpdatePageLayoutDto } from './dto/page-layout.dto';
|
||||
import { CreatePageLayoutDto, UpdatePageLayoutDto, PageLayoutType } from './dto/page-layout.dto';
|
||||
|
||||
@Injectable()
|
||||
export class PageLayoutService {
|
||||
constructor(private tenantDbService: TenantDatabaseService) {}
|
||||
|
||||
async create(tenantId: string, createDto: CreatePageLayoutDto) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
const layoutType = createDto.layoutType || 'detail';
|
||||
|
||||
// If this layout is set as default, unset other defaults for the same object
|
||||
// If this layout is set as default, unset other defaults for the same object and layout type
|
||||
if (createDto.isDefault) {
|
||||
await knex('page_layouts')
|
||||
.where({ object_id: createDto.objectId })
|
||||
.where({ object_id: createDto.objectId, layout_type: layoutType })
|
||||
.update({ is_default: false });
|
||||
}
|
||||
|
||||
const [id] = await knex('page_layouts').insert({
|
||||
name: createDto.name,
|
||||
object_id: createDto.objectId,
|
||||
layout_type: layoutType,
|
||||
is_default: createDto.isDefault || false,
|
||||
layout_config: JSON.stringify(createDto.layoutConfig),
|
||||
description: createDto.description || null,
|
||||
@@ -29,8 +31,8 @@ export class PageLayoutService {
|
||||
return result;
|
||||
}
|
||||
|
||||
async findAll(tenantId: string, objectId?: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
async findAll(tenantId: string, objectId?: string, layoutType?: PageLayoutType) {
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
let query = knex('page_layouts');
|
||||
|
||||
@@ -38,12 +40,16 @@ export class PageLayoutService {
|
||||
query = query.where({ object_id: objectId });
|
||||
}
|
||||
|
||||
if (layoutType) {
|
||||
query = query.where({ layout_type: layoutType });
|
||||
}
|
||||
|
||||
const layouts = await query.orderByRaw('is_default DESC, name ASC');
|
||||
return layouts;
|
||||
}
|
||||
|
||||
async findOne(tenantId: string, id: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
const layout = await knex('page_layouts').where({ id }).first();
|
||||
|
||||
@@ -54,27 +60,26 @@ export class PageLayoutService {
|
||||
return layout;
|
||||
}
|
||||
|
||||
async findDefaultByObject(tenantId: string, objectId: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
async findDefaultByObject(tenantId: string, objectId: string, layoutType: PageLayoutType = 'detail') {
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
const layout = await knex('page_layouts')
|
||||
.where({ object_id: objectId, is_default: true })
|
||||
.where({ object_id: objectId, is_default: true, layout_type: layoutType })
|
||||
.first();
|
||||
|
||||
return layout || null;
|
||||
}
|
||||
|
||||
async update(tenantId: string, id: string, updateDto: UpdatePageLayoutDto) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
// Check if layout exists
|
||||
await this.findOne(tenantId, id);
|
||||
|
||||
// If setting as default, unset other defaults for the same object
|
||||
if (updateDto.isDefault) {
|
||||
const layout = await this.findOne(tenantId, id);
|
||||
|
||||
// If setting as default, unset other defaults for the same object and layout type
|
||||
if (updateDto.isDefault) {
|
||||
await knex('page_layouts')
|
||||
.where({ object_id: layout.object_id })
|
||||
.where({ object_id: layout.object_id, layout_type: layout.layout_type })
|
||||
.whereNot({ id })
|
||||
.update({ is_default: false });
|
||||
}
|
||||
@@ -107,7 +112,7 @@ export class PageLayoutService {
|
||||
}
|
||||
|
||||
async remove(tenantId: string, id: string) {
|
||||
const knex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
await this.findOne(tenantId, id);
|
||||
|
||||
|
||||
199
backend/src/rbac/ability.factory.ts
Normal file
199
backend/src/rbac/ability.factory.ts
Normal file
@@ -0,0 +1,199 @@
|
||||
import { AbilityBuilder, PureAbility, AbilityClass } from '@casl/ability';
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import { User } from '../models/user.model';
|
||||
import { RoleObjectPermission } from '../models/role-object-permission.model';
|
||||
import { RoleFieldPermission } from '../models/role-field-permission.model';
|
||||
import { RecordShare } from '../models/record-share.model';
|
||||
|
||||
// Define action types
|
||||
export type Action = 'create' | 'read' | 'update' | 'delete' | 'view_all' | 'modify_all';
|
||||
|
||||
// Define subject types - can be string (object API name) or actual object with fields
|
||||
export type Subject = string | { objectApiName: string; ownerId?: string; id?: string; [key: string]: any };
|
||||
|
||||
// Define field actions
|
||||
export type FieldAction = 'read' | 'edit';
|
||||
|
||||
export type AppAbility = PureAbility<[Action, Subject], { field?: string }>;
|
||||
|
||||
@Injectable()
|
||||
export class AbilityFactory {
|
||||
/**
|
||||
* Build CASL ability for a user based on their roles and permissions
|
||||
* This aggregates permissions from all roles the user has
|
||||
*/
|
||||
async defineAbilityFor(
|
||||
user: User & { roles?: Array<{ objectPermissions?: RoleObjectPermission[]; fieldPermissions?: RoleFieldPermission[] }> },
|
||||
recordShares?: RecordShare[],
|
||||
): Promise<AppAbility> {
|
||||
const { can, cannot, build } = new AbilityBuilder<AppAbility>(PureAbility as AbilityClass<AppAbility>);
|
||||
|
||||
if (!user.roles || user.roles.length === 0) {
|
||||
// No roles = no permissions
|
||||
return build();
|
||||
}
|
||||
|
||||
// Aggregate object permissions from all roles
|
||||
const objectPermissionsMap = new Map<string, {
|
||||
canCreate: boolean;
|
||||
canRead: boolean;
|
||||
canEdit: boolean;
|
||||
canDelete: boolean;
|
||||
canViewAll: boolean;
|
||||
canModifyAll: boolean;
|
||||
}>();
|
||||
|
||||
// Aggregate field permissions from all roles
|
||||
const fieldPermissionsMap = new Map<string, {
|
||||
canRead: boolean;
|
||||
canEdit: boolean;
|
||||
}>();
|
||||
|
||||
// Process all roles
|
||||
for (const role of user.roles) {
|
||||
// Aggregate object permissions
|
||||
if (role.objectPermissions) {
|
||||
for (const perm of role.objectPermissions) {
|
||||
const existing = objectPermissionsMap.get(perm.objectDefinitionId) || {
|
||||
canCreate: false,
|
||||
canRead: false,
|
||||
canEdit: false,
|
||||
canDelete: false,
|
||||
canViewAll: false,
|
||||
canModifyAll: false,
|
||||
};
|
||||
|
||||
// Union of permissions (if any role grants it, user has it)
|
||||
objectPermissionsMap.set(perm.objectDefinitionId, {
|
||||
canCreate: existing.canCreate || perm.canCreate,
|
||||
canRead: existing.canRead || perm.canRead,
|
||||
canEdit: existing.canEdit || perm.canEdit,
|
||||
canDelete: existing.canDelete || perm.canDelete,
|
||||
canViewAll: existing.canViewAll || perm.canViewAll,
|
||||
canModifyAll: existing.canModifyAll || perm.canModifyAll,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Aggregate field permissions
|
||||
if (role.fieldPermissions) {
|
||||
for (const perm of role.fieldPermissions) {
|
||||
const existing = fieldPermissionsMap.get(perm.fieldDefinitionId) || {
|
||||
canRead: false,
|
||||
canEdit: false,
|
||||
};
|
||||
|
||||
fieldPermissionsMap.set(perm.fieldDefinitionId, {
|
||||
canRead: existing.canRead || perm.canRead,
|
||||
canEdit: existing.canEdit || perm.canEdit,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Convert aggregated permissions to CASL rules
|
||||
for (const [objectId, perms] of objectPermissionsMap) {
|
||||
// Create permission
|
||||
if (perms.canCreate) {
|
||||
can('create', objectId);
|
||||
}
|
||||
|
||||
// Read permission
|
||||
if (perms.canRead) {
|
||||
can('read', objectId);
|
||||
}
|
||||
|
||||
// View all permission (can see all records regardless of ownership)
|
||||
if (perms.canViewAll) {
|
||||
can('view_all', objectId);
|
||||
}
|
||||
|
||||
// Edit permission
|
||||
if (perms.canEdit) {
|
||||
can('update', objectId);
|
||||
}
|
||||
|
||||
// Modify all permission (can edit all records regardless of ownership)
|
||||
if (perms.canModifyAll) {
|
||||
can('modify_all', objectId);
|
||||
}
|
||||
|
||||
// Delete permission
|
||||
if (perms.canDelete) {
|
||||
can('delete', objectId);
|
||||
}
|
||||
}
|
||||
|
||||
// Add record sharing permissions
|
||||
if (recordShares) {
|
||||
for (const share of recordShares) {
|
||||
// Only add if share is active (not expired, not revoked)
|
||||
const now = new Date();
|
||||
const isExpired = share.expiresAt && share.expiresAt < now;
|
||||
const isRevoked = share.revokedAt !== null;
|
||||
|
||||
if (!isExpired && !isRevoked) {
|
||||
// Note: Record-level sharing will be checked in authorization service
|
||||
// CASL abilities are primarily for object-level permissions
|
||||
// Individual record access is validated in applyScopeToQuery
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return build();
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if user can access a specific field
|
||||
* Returns true if user has permission or if no restriction exists
|
||||
*/
|
||||
canAccessField(
|
||||
fieldDefinitionId: string,
|
||||
action: FieldAction,
|
||||
user: User & { roles?: Array<{ fieldPermissions?: RoleFieldPermission[] }> },
|
||||
): boolean {
|
||||
if (!user.roles || user.roles.length === 0) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Collect all field permissions from all roles
|
||||
const allFieldPermissions: RoleFieldPermission[] = [];
|
||||
for (const role of user.roles) {
|
||||
if (role.fieldPermissions) {
|
||||
allFieldPermissions.push(...role.fieldPermissions);
|
||||
}
|
||||
}
|
||||
|
||||
// If there are NO field permissions configured at all, allow by default
|
||||
if (allFieldPermissions.length === 0) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// If field permissions exist, check for explicit grants (union of all roles)
|
||||
for (const role of user.roles) {
|
||||
if (role.fieldPermissions) {
|
||||
const fieldPerm = role.fieldPermissions.find(fp => fp.fieldDefinitionId === fieldDefinitionId);
|
||||
if (fieldPerm) {
|
||||
if (action === 'read' && fieldPerm.canRead) return true;
|
||||
if (action === 'edit' && fieldPerm.canEdit) return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// No explicit rule for this field but other field permissions exist.
|
||||
// Default to allow so new fields don't get silently stripped and fail validation.
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Filter fields based on user permissions
|
||||
* Returns array of field IDs the user can access with the specified action
|
||||
*/
|
||||
filterFields(
|
||||
fieldDefinitionIds: string[],
|
||||
action: FieldAction,
|
||||
user: User & { roles?: Array<{ fieldPermissions?: RoleFieldPermission[] }> },
|
||||
): string[] {
|
||||
return fieldDefinitionIds.filter(fieldId => this.canAccessField(fieldId, action, user));
|
||||
}
|
||||
}
|
||||
282
backend/src/rbac/authorization.service.ts
Normal file
282
backend/src/rbac/authorization.service.ts
Normal file
@@ -0,0 +1,282 @@
|
||||
import { Injectable, ForbiddenException } from '@nestjs/common';
|
||||
import { Knex } from 'knex';
|
||||
import { User } from '../models/user.model';
|
||||
import { ObjectDefinition } from '../models/object-definition.model';
|
||||
import { FieldDefinition } from '../models/field-definition.model';
|
||||
import { RecordShare } from '../models/record-share.model';
|
||||
import { AbilityFactory, AppAbility, Action } from './ability.factory';
|
||||
import { DynamicModelFactory } from '../object/models/dynamic-model.factory';
|
||||
import { subject } from '@casl/ability';
|
||||
|
||||
@Injectable()
|
||||
export class AuthorizationService {
|
||||
constructor(private abilityFactory: AbilityFactory) {}
|
||||
|
||||
/**
|
||||
* Apply authorization scope to a query based on OWD and user permissions
|
||||
* This determines which records the user can see
|
||||
* Modifies the query in place and returns void
|
||||
*/
|
||||
async applyScopeToQuery<T = any>(
|
||||
query: any, // Accept both Knex and Objection query builders
|
||||
objectDef: ObjectDefinition,
|
||||
user: User & { roles?: any[] },
|
||||
action: Action,
|
||||
knex: Knex,
|
||||
): Promise<void> {
|
||||
// Get user's ability
|
||||
const recordShares = await this.getActiveRecordShares(objectDef.id, user.id, knex);
|
||||
const ability = await this.abilityFactory.defineAbilityFor(user, recordShares);
|
||||
|
||||
// Check if user has the base permission for this action
|
||||
// Use object ID, not API name, since permissions are stored by object ID
|
||||
if (!ability.can(action, objectDef.id)) {
|
||||
// No permission at all - return empty result
|
||||
query.where(knex.raw('1 = 0'));
|
||||
return;
|
||||
}
|
||||
|
||||
// Check special permissions
|
||||
const hasViewAll = ability.can('view_all', objectDef.id);
|
||||
const hasModifyAll = ability.can('modify_all', objectDef.id);
|
||||
|
||||
// If user has view_all or modify_all, they can see all records
|
||||
if (hasViewAll || hasModifyAll) {
|
||||
// No filtering needed
|
||||
return;
|
||||
}
|
||||
|
||||
// Apply OWD (Org-Wide Default) restrictions
|
||||
switch (objectDef.orgWideDefault) {
|
||||
case 'public_read_write':
|
||||
// Everyone can see all records
|
||||
return;
|
||||
|
||||
case 'public_read':
|
||||
// Everyone can see all records (write operations checked separately)
|
||||
return;
|
||||
|
||||
case 'private':
|
||||
default:
|
||||
// Only owner and explicitly shared records
|
||||
await this.applyPrivateScope(query, objectDef, user, recordShares, knex);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply private scope: owner + shared records
|
||||
*/
|
||||
private async applyPrivateScope<T = any>(
|
||||
query: any, // Accept both Knex and Objection query builders
|
||||
objectDef: ObjectDefinition,
|
||||
user: User,
|
||||
recordShares: RecordShare[],
|
||||
knex: Knex,
|
||||
): Promise<void> {
|
||||
const tableName = this.getTableName(objectDef.apiName);
|
||||
|
||||
// Check if table has ownerId column
|
||||
const hasOwner = await knex.schema.hasColumn(tableName, 'ownerId');
|
||||
|
||||
if (!hasOwner && recordShares.length === 0) {
|
||||
// No ownership and no shares - user can't see anything
|
||||
query.where(knex.raw('1 = 0'));
|
||||
return;
|
||||
}
|
||||
|
||||
// Build conditions: ownerId = user OR record shared with user
|
||||
query.where((builder) => {
|
||||
if (hasOwner) {
|
||||
builder.orWhere(`${tableName}.ownerId`, user.id);
|
||||
}
|
||||
|
||||
if (recordShares.length > 0) {
|
||||
const sharedRecordIds = recordShares.map(share => share.recordId);
|
||||
builder.orWhereIn(`${tableName}.id`, sharedRecordIds);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if user can perform action on a specific record
|
||||
*/
|
||||
async canPerformAction(
|
||||
action: Action,
|
||||
objectDef: ObjectDefinition,
|
||||
record: any,
|
||||
user: User & { roles?: any[] },
|
||||
knex: Knex,
|
||||
): Promise<boolean> {
|
||||
const recordShares = await this.getActiveRecordShares(objectDef.id, user.id, knex);
|
||||
const ability = await this.abilityFactory.defineAbilityFor(user, recordShares);
|
||||
|
||||
// Check base permission - use object ID not API name
|
||||
if (!ability.can(action, objectDef.id)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Check special permissions - use object ID not API name
|
||||
const hasViewAll = ability.can('view_all', objectDef.id);
|
||||
const hasModifyAll = ability.can('modify_all', objectDef.id);
|
||||
|
||||
// canViewAll only grants read access to all records
|
||||
if (action === 'read' && hasViewAll) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// canModifyAll grants edit/delete access to all records
|
||||
if ((action === 'update' || action === 'delete') && hasModifyAll) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// Check OWD
|
||||
switch (objectDef.orgWideDefault) {
|
||||
case 'public_read_write':
|
||||
return true;
|
||||
|
||||
case 'public_read':
|
||||
if (action === 'read') return true;
|
||||
// For write actions, check ownership
|
||||
return record.ownerId === user.id;
|
||||
|
||||
case 'private':
|
||||
default:
|
||||
// Check ownership
|
||||
if (record.ownerId === user.id) return true;
|
||||
|
||||
// Check if record is shared with user
|
||||
const share = recordShares.find(s => s.recordId === record.id);
|
||||
if (share) {
|
||||
if (action === 'read' && share.accessLevel.canRead) return true;
|
||||
if (action === 'update' && share.accessLevel.canEdit) return true;
|
||||
if (action === 'delete' && share.accessLevel.canDelete) return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Filter data based on field-level permissions
|
||||
* Removes fields the user cannot read
|
||||
*/
|
||||
async filterReadableFields(
|
||||
data: any,
|
||||
fields: FieldDefinition[],
|
||||
user: User & { roles?: any[] },
|
||||
): Promise<any> {
|
||||
const filtered: any = {};
|
||||
|
||||
// Always include id - it's required for navigation and record identification
|
||||
if (data.id !== undefined) {
|
||||
filtered.id = data.id;
|
||||
}
|
||||
|
||||
for (const field of fields) {
|
||||
if (this.abilityFactory.canAccessField(field.id, 'read', user)) {
|
||||
if (data[field.apiName] !== undefined) {
|
||||
filtered[field.apiName] = data[field.apiName];
|
||||
}
|
||||
|
||||
// For lookup fields, also include the related object (e.g., ownerId -> owner)
|
||||
if (field.type === 'LOOKUP') {
|
||||
const relationName = DynamicModelFactory.getRelationName(field.apiName);
|
||||
if (data[relationName] !== undefined) {
|
||||
filtered[relationName] = data[relationName];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return filtered;
|
||||
}
|
||||
|
||||
/**
|
||||
* Filter data based on field-level permissions
|
||||
* Removes fields the user cannot edit
|
||||
*/
|
||||
async filterEditableFields(
|
||||
data: any,
|
||||
fields: FieldDefinition[],
|
||||
user: User & { roles?: any[] },
|
||||
): Promise<any> {
|
||||
const filtered: any = {};
|
||||
|
||||
for (const field of fields) {
|
||||
if (this.abilityFactory.canAccessField(field.id, 'edit', user)) {
|
||||
if (data[field.apiName] !== undefined) {
|
||||
filtered[field.apiName] = data[field.apiName];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return filtered;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get active record shares for a user on an object
|
||||
*/
|
||||
private async getActiveRecordShares(
|
||||
objectDefinitionId: string,
|
||||
userId: string,
|
||||
knex: Knex,
|
||||
): Promise<RecordShare[]> {
|
||||
const now = new Date();
|
||||
|
||||
return await RecordShare.query(knex)
|
||||
.where('objectDefinitionId', objectDefinitionId)
|
||||
.where('granteeUserId', userId)
|
||||
.whereNull('revokedAt')
|
||||
.where((builder) => {
|
||||
builder.whereNull('expiresAt').orWhere('expiresAt', '>', now);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if user has permission to create records
|
||||
*/
|
||||
async canCreate(
|
||||
objectDef: ObjectDefinition,
|
||||
user: User & { roles?: any[] },
|
||||
): Promise<boolean> {
|
||||
const ability = await this.abilityFactory.defineAbilityFor(user, []);
|
||||
return ability.can('create', objectDef.id);
|
||||
}
|
||||
|
||||
/**
|
||||
* Throw exception if user cannot perform action
|
||||
*/
|
||||
async assertCanPerformAction(
|
||||
action: Action,
|
||||
objectDef: ObjectDefinition,
|
||||
record: any,
|
||||
user: User & { roles?: any[] },
|
||||
knex: Knex,
|
||||
): Promise<void> {
|
||||
const can = await this.canPerformAction(action, objectDef, record, user, knex);
|
||||
if (!can) {
|
||||
throw new ForbiddenException(`You do not have permission to ${action} this record`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get table name from API name
|
||||
*/
|
||||
private getTableName(apiName: string): string {
|
||||
// Convert CamelCase to snake_case and pluralize
|
||||
const snakeCase = apiName
|
||||
.replace(/([A-Z])/g, '_$1')
|
||||
.toLowerCase()
|
||||
.replace(/^_/, '');
|
||||
|
||||
// Simple pluralization
|
||||
if (snakeCase.endsWith('y')) {
|
||||
return snakeCase.slice(0, -1) + 'ies';
|
||||
} else if (snakeCase.endsWith('s')) {
|
||||
return snakeCase;
|
||||
} else {
|
||||
return snakeCase + 's';
|
||||
}
|
||||
}
|
||||
}
|
||||
19
backend/src/rbac/dto/create-record-share.dto.ts
Normal file
19
backend/src/rbac/dto/create-record-share.dto.ts
Normal file
@@ -0,0 +1,19 @@
|
||||
import { IsString, IsBoolean, IsOptional, IsDateString } from 'class-validator';
|
||||
|
||||
export class CreateRecordShareDto {
|
||||
@IsString()
|
||||
granteeUserId: string;
|
||||
|
||||
@IsBoolean()
|
||||
canRead: boolean;
|
||||
|
||||
@IsBoolean()
|
||||
canEdit: boolean;
|
||||
|
||||
@IsBoolean()
|
||||
canDelete: boolean;
|
||||
|
||||
@IsOptional()
|
||||
@IsDateString()
|
||||
expiresAt?: string;
|
||||
}
|
||||
@@ -1,8 +1,16 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { RbacService } from './rbac.service';
|
||||
import { AbilityFactory } from './ability.factory';
|
||||
import { AuthorizationService } from './authorization.service';
|
||||
import { SetupRolesController } from './setup-roles.controller';
|
||||
import { SetupUsersController } from './setup-users.controller';
|
||||
import { RecordSharingController } from './record-sharing.controller';
|
||||
import { TenantModule } from '../tenant/tenant.module';
|
||||
|
||||
@Module({
|
||||
providers: [RbacService],
|
||||
exports: [RbacService],
|
||||
imports: [TenantModule],
|
||||
controllers: [SetupRolesController, SetupUsersController, RecordSharingController],
|
||||
providers: [RbacService, AbilityFactory, AuthorizationService],
|
||||
exports: [RbacService, AbilityFactory, AuthorizationService],
|
||||
})
|
||||
export class RbacModule {}
|
||||
|
||||
350
backend/src/rbac/record-sharing.controller.ts
Normal file
350
backend/src/rbac/record-sharing.controller.ts
Normal file
@@ -0,0 +1,350 @@
|
||||
import {
|
||||
Controller,
|
||||
Get,
|
||||
Post,
|
||||
Delete,
|
||||
Param,
|
||||
Body,
|
||||
UseGuards,
|
||||
ForbiddenException,
|
||||
} from '@nestjs/common';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { TenantId } from '../tenant/tenant.decorator';
|
||||
import { CurrentUser } from '../auth/current-user.decorator';
|
||||
import { TenantDatabaseService } from '../tenant/tenant-database.service';
|
||||
import { RecordShare } from '../models/record-share.model';
|
||||
import { ObjectDefinition } from '../models/object-definition.model';
|
||||
import { User } from '../models/user.model';
|
||||
import { AuthorizationService } from './authorization.service';
|
||||
import { CreateRecordShareDto } from './dto/create-record-share.dto';
|
||||
|
||||
@Controller('runtime/objects/:objectApiName/records/:recordId/shares')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
export class RecordSharingController {
|
||||
constructor(
|
||||
private tenantDbService: TenantDatabaseService,
|
||||
private authService: AuthorizationService,
|
||||
) {}
|
||||
|
||||
@Get()
|
||||
async getRecordShares(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Param('recordId') recordId: string,
|
||||
@CurrentUser() currentUser: any,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Get object definition
|
||||
const objectDef = await ObjectDefinition.query(knex)
|
||||
.findOne({ apiName: objectApiName });
|
||||
|
||||
if (!objectDef) {
|
||||
throw new Error('Object not found');
|
||||
}
|
||||
|
||||
// Get the record to check ownership
|
||||
const tableName = this.getTableName(
|
||||
objectDef.apiName,
|
||||
objectDef.label,
|
||||
objectDef.pluralLabel,
|
||||
);
|
||||
const record = await knex(tableName)
|
||||
.where({ id: recordId })
|
||||
.first();
|
||||
|
||||
if (!record) {
|
||||
throw new Error('Record not found');
|
||||
}
|
||||
|
||||
// Only owner can view shares
|
||||
if (record.ownerId !== currentUser.userId) {
|
||||
// Check if user has modify all permission
|
||||
const user: any = await User.query(knex)
|
||||
.findById(currentUser.userId)
|
||||
.withGraphFetched('roles.objectPermissions');
|
||||
|
||||
if (!user) {
|
||||
throw new ForbiddenException('User not found');
|
||||
}
|
||||
|
||||
const hasModifyAll = user.roles?.some(role =>
|
||||
role.objectPermissions?.some(
|
||||
perm => perm.objectDefinitionId === objectDef.id && perm.canModifyAll
|
||||
)
|
||||
);
|
||||
|
||||
if (!hasModifyAll) {
|
||||
throw new ForbiddenException('Only the record owner or users with Modify All permission can view shares');
|
||||
}
|
||||
}
|
||||
|
||||
// Get all active shares for this record
|
||||
const shares = await RecordShare.query(knex)
|
||||
.where({ objectDefinitionId: objectDef.id, recordId })
|
||||
.whereNull('revokedAt')
|
||||
.where(builder => {
|
||||
builder.whereNull('expiresAt').orWhere('expiresAt', '>', new Date());
|
||||
})
|
||||
.withGraphFetched('[granteeUser]')
|
||||
.orderBy('createdAt', 'desc');
|
||||
|
||||
return shares;
|
||||
}
|
||||
|
||||
@Post()
|
||||
async createRecordShare(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Param('recordId') recordId: string,
|
||||
@CurrentUser() currentUser: any,
|
||||
@Body() data: CreateRecordShareDto,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Get object definition
|
||||
const objectDef = await ObjectDefinition.query(knex)
|
||||
.findOne({ apiName: objectApiName });
|
||||
|
||||
if (!objectDef) {
|
||||
throw new Error('Object not found');
|
||||
}
|
||||
|
||||
// Get the record to check ownership
|
||||
const tableName = this.getTableName(
|
||||
objectDef.apiName,
|
||||
objectDef.label,
|
||||
objectDef.pluralLabel,
|
||||
);
|
||||
const record = await knex(tableName)
|
||||
.where({ id: recordId })
|
||||
.first();
|
||||
|
||||
if (!record) {
|
||||
throw new Error('Record not found');
|
||||
}
|
||||
|
||||
// Check if user can share - either owner or has modify permissions
|
||||
const canShare = await this.canUserShareRecord(
|
||||
currentUser.userId,
|
||||
record,
|
||||
objectDef,
|
||||
knex,
|
||||
);
|
||||
|
||||
if (!canShare) {
|
||||
throw new ForbiddenException('You do not have permission to share this record');
|
||||
}
|
||||
|
||||
// Cannot share with self
|
||||
if (data.granteeUserId === currentUser.userId) {
|
||||
throw new Error('Cannot share record with yourself');
|
||||
}
|
||||
|
||||
// Check if share already exists
|
||||
const existingShare = await RecordShare.query(knex)
|
||||
.where({
|
||||
objectDefinitionId: objectDef.id,
|
||||
recordId,
|
||||
granteeUserId: data.granteeUserId,
|
||||
})
|
||||
.whereNull('revokedAt')
|
||||
.first();
|
||||
|
||||
if (existingShare) {
|
||||
// Update existing share
|
||||
const updated = await RecordShare.query(knex)
|
||||
.patchAndFetchById(existingShare.id, {
|
||||
accessLevel: {
|
||||
canRead: data.canRead,
|
||||
canEdit: data.canEdit,
|
||||
canDelete: data.canDelete,
|
||||
},
|
||||
// Convert ISO string to MySQL datetime format
|
||||
expiresAt: data.expiresAt
|
||||
? knex.raw('?', [new Date(data.expiresAt).toISOString().slice(0, 19).replace('T', ' ')])
|
||||
: null,
|
||||
} as any);
|
||||
|
||||
return RecordShare.query(knex)
|
||||
.findById(updated.id)
|
||||
.withGraphFetched('[granteeUser]');
|
||||
}
|
||||
|
||||
// Create new share
|
||||
const share = await RecordShare.query(knex).insertAndFetch({
|
||||
objectDefinitionId: objectDef.id,
|
||||
recordId,
|
||||
granteeUserId: data.granteeUserId,
|
||||
grantedByUserId: currentUser.userId,
|
||||
accessLevel: {
|
||||
canRead: data.canRead,
|
||||
canEdit: data.canEdit,
|
||||
canDelete: data.canDelete,
|
||||
},
|
||||
// Convert ISO string to MySQL datetime format: YYYY-MM-DD HH:MM:SS
|
||||
expiresAt: data.expiresAt
|
||||
? knex.raw('?', [new Date(data.expiresAt).toISOString().slice(0, 19).replace('T', ' ')])
|
||||
: null,
|
||||
} as any);
|
||||
|
||||
return RecordShare.query(knex)
|
||||
.findById(share.id)
|
||||
.withGraphFetched('[granteeUser]');
|
||||
}
|
||||
|
||||
@Delete(':shareId')
|
||||
async deleteRecordShare(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
@Param('recordId') recordId: string,
|
||||
@Param('shareId') shareId: string,
|
||||
@CurrentUser() currentUser: any,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Get object definition
|
||||
const objectDef = await ObjectDefinition.query(knex)
|
||||
.findOne({ apiName: objectApiName });
|
||||
|
||||
if (!objectDef) {
|
||||
throw new Error('Object not found');
|
||||
}
|
||||
|
||||
// Get the record to check ownership
|
||||
const tableName = this.getTableName(
|
||||
objectDef.apiName,
|
||||
objectDef.label,
|
||||
objectDef.pluralLabel,
|
||||
);
|
||||
const record = await knex(tableName)
|
||||
.where({ id: recordId })
|
||||
.first();
|
||||
|
||||
if (!record) {
|
||||
throw new Error('Record not found');
|
||||
}
|
||||
|
||||
// Only owner can revoke shares
|
||||
if (record.ownerId !== currentUser.userId) {
|
||||
// Check if user has modify all permission
|
||||
const user: any = await User.query(knex)
|
||||
.findById(currentUser.userId)
|
||||
.withGraphFetched('roles.objectPermissions');
|
||||
|
||||
if (!user) {
|
||||
throw new ForbiddenException('User not found');
|
||||
}
|
||||
|
||||
const hasModifyAll = user.roles?.some(role =>
|
||||
role.objectPermissions?.some(
|
||||
perm => perm.objectDefinitionId === objectDef.id && perm.canModifyAll
|
||||
)
|
||||
);
|
||||
|
||||
if (!hasModifyAll) {
|
||||
throw new ForbiddenException('Only the record owner or users with Modify All permission can revoke shares');
|
||||
}
|
||||
}
|
||||
|
||||
// Revoke the share (soft delete)
|
||||
await RecordShare.query(knex)
|
||||
.patchAndFetchById(shareId, {
|
||||
revokedAt: knex.fn.now() as any,
|
||||
});
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
private async canUserShareRecord(
|
||||
userId: string,
|
||||
record: any,
|
||||
objectDef: ObjectDefinition,
|
||||
knex: any,
|
||||
): Promise<boolean> {
|
||||
// Owner can always share
|
||||
if (record.ownerId === userId) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// Check if user has modify all or edit permissions
|
||||
const user: any = await User.query(knex)
|
||||
.findById(userId)
|
||||
.withGraphFetched('roles.objectPermissions');
|
||||
|
||||
if (!user) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Check for canModifyAll permission
|
||||
const hasModifyAll = user.roles?.some(role =>
|
||||
role.objectPermissions?.some(
|
||||
perm => perm.objectDefinitionId === objectDef.id && perm.canModifyAll
|
||||
)
|
||||
);
|
||||
|
||||
if (hasModifyAll) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// Check for canEdit permission (user needs edit to share)
|
||||
const hasEdit = user.roles?.some(role =>
|
||||
role.objectPermissions?.some(
|
||||
perm => perm.objectDefinitionId === objectDef.id && perm.canEdit
|
||||
)
|
||||
);
|
||||
|
||||
// If user has edit permission, check if they can actually edit this record
|
||||
// by using the authorization service
|
||||
if (hasEdit) {
|
||||
try {
|
||||
await this.authService.assertCanPerformAction(
|
||||
'update',
|
||||
objectDef,
|
||||
record,
|
||||
user,
|
||||
knex,
|
||||
);
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
private getTableName(apiName: string, objectLabel?: string, pluralLabel?: string): string {
|
||||
const toSnakePlural = (source: string): string => {
|
||||
const cleaned = source.replace(/[\s-]+/g, '_');
|
||||
const snake = cleaned
|
||||
.replace(/([a-z0-9])([A-Z])/g, '$1_$2')
|
||||
.replace(/__+/g, '_')
|
||||
.toLowerCase()
|
||||
.replace(/^_/, '');
|
||||
|
||||
if (snake.endsWith('y')) return `${snake.slice(0, -1)}ies`;
|
||||
if (snake.endsWith('s')) return snake;
|
||||
return `${snake}s`;
|
||||
};
|
||||
|
||||
const fromApi = toSnakePlural(apiName);
|
||||
const fromLabel = objectLabel ? toSnakePlural(objectLabel) : null;
|
||||
const fromPlural = pluralLabel ? toSnakePlural(pluralLabel) : null;
|
||||
|
||||
if (fromLabel && fromLabel.includes('_') && !fromApi.includes('_')) {
|
||||
return fromLabel;
|
||||
}
|
||||
if (fromPlural && fromPlural.includes('_') && !fromApi.includes('_')) {
|
||||
return fromPlural;
|
||||
}
|
||||
|
||||
if (fromLabel && fromLabel !== fromApi) return fromLabel;
|
||||
if (fromPlural && fromPlural !== fromApi) return fromPlural;
|
||||
|
||||
return fromApi;
|
||||
}
|
||||
}
|
||||
141
backend/src/rbac/setup-roles.controller.ts
Normal file
141
backend/src/rbac/setup-roles.controller.ts
Normal file
@@ -0,0 +1,141 @@
|
||||
import {
|
||||
Controller,
|
||||
Get,
|
||||
Post,
|
||||
Patch,
|
||||
Delete,
|
||||
Param,
|
||||
Body,
|
||||
UseGuards,
|
||||
} from '@nestjs/common';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { TenantId } from '../tenant/tenant.decorator';
|
||||
import { TenantDatabaseService } from '../tenant/tenant-database.service';
|
||||
import { Role } from '../models/role.model';
|
||||
|
||||
@Controller('setup/roles')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
export class SetupRolesController {
|
||||
constructor(private tenantDbService: TenantDatabaseService) {}
|
||||
|
||||
@Get()
|
||||
async getRoles(@TenantId() tenantId: string) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
return await Role.query(knex).select('*').orderBy('name', 'asc');
|
||||
}
|
||||
|
||||
@Get(':id')
|
||||
async getRole(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('id') id: string,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
return await Role.query(knex).findById(id).withGraphFetched('users');
|
||||
}
|
||||
|
||||
@Post()
|
||||
async createRole(
|
||||
@TenantId() tenantId: string,
|
||||
@Body() data: { name: string; description?: string; guardName?: string },
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const role = await Role.query(knex).insert({
|
||||
name: data.name,
|
||||
description: data.description,
|
||||
guardName: data.guardName || 'tenant',
|
||||
});
|
||||
|
||||
return role;
|
||||
}
|
||||
|
||||
@Patch(':id')
|
||||
async updateRole(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('id') id: string,
|
||||
@Body() data: { name?: string; description?: string; guardName?: string },
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const updateData: any = {};
|
||||
|
||||
if (data.name) updateData.name = data.name;
|
||||
if (data.description !== undefined) updateData.description = data.description;
|
||||
if (data.guardName) updateData.guardName = data.guardName;
|
||||
|
||||
const role = await Role.query(knex).patchAndFetchById(id, updateData);
|
||||
return role;
|
||||
}
|
||||
|
||||
@Delete(':id')
|
||||
async deleteRole(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('id') id: string,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Delete role user assignments first
|
||||
await knex('user_roles').where({ roleId: id }).delete();
|
||||
|
||||
// Delete role permissions
|
||||
await knex('role_permissions').where({ roleId: id }).delete();
|
||||
await knex('role_object_permissions').where({ roleId: id }).delete();
|
||||
|
||||
// Delete the role
|
||||
await Role.query(knex).deleteById(id);
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
@Post(':roleId/users')
|
||||
async addUserToRole(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('roleId') roleId: string,
|
||||
@Body() data: { userId: string },
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Check if assignment already exists
|
||||
const existing = await knex('user_roles')
|
||||
.where({ userId: data.userId, roleId })
|
||||
.first();
|
||||
|
||||
if (existing) {
|
||||
return { success: true, message: 'User already assigned' };
|
||||
}
|
||||
|
||||
await knex('user_roles').insert({
|
||||
id: knex.raw('(UUID())'),
|
||||
userId: data.userId,
|
||||
roleId,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
@Delete(':roleId/users/:userId')
|
||||
async removeUserFromRole(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('roleId') roleId: string,
|
||||
@Param('userId') userId: string,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
await knex('user_roles')
|
||||
.where({ userId, roleId })
|
||||
.delete();
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
}
|
||||
148
backend/src/rbac/setup-users.controller.ts
Normal file
148
backend/src/rbac/setup-users.controller.ts
Normal file
@@ -0,0 +1,148 @@
|
||||
import {
|
||||
Controller,
|
||||
Get,
|
||||
Post,
|
||||
Patch,
|
||||
Delete,
|
||||
Param,
|
||||
Body,
|
||||
UseGuards,
|
||||
} from '@nestjs/common';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { TenantId } from '../tenant/tenant.decorator';
|
||||
import { TenantDatabaseService } from '../tenant/tenant-database.service';
|
||||
import { User } from '../models/user.model';
|
||||
import * as bcrypt from 'bcrypt';
|
||||
|
||||
@Controller('setup/users')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
export class SetupUsersController {
|
||||
constructor(private tenantDbService: TenantDatabaseService) {}
|
||||
|
||||
@Get()
|
||||
async getUsers(@TenantId() tenantId: string) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
return await User.query(knex).withGraphFetched('roles');
|
||||
}
|
||||
|
||||
@Get(':id')
|
||||
async getUser(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('id') id: string,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
return await User.query(knex).findById(id).withGraphFetched('roles');
|
||||
}
|
||||
|
||||
@Post()
|
||||
async createUser(
|
||||
@TenantId() tenantId: string,
|
||||
@Body() data: { email: string; password: string; firstName?: string; lastName?: string; alias?: string },
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Hash password
|
||||
const hashedPassword = await bcrypt.hash(data.password, 10);
|
||||
|
||||
const user = await User.query(knex).insert({
|
||||
email: data.email,
|
||||
password: hashedPassword,
|
||||
firstName: data.firstName,
|
||||
lastName: data.lastName,
|
||||
alias: data.alias,
|
||||
isActive: true,
|
||||
});
|
||||
|
||||
return user;
|
||||
}
|
||||
|
||||
@Patch(':id')
|
||||
async updateUser(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('id') id: string,
|
||||
@Body() data: { email?: string; password?: string; firstName?: string; lastName?: string; alias?: string },
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const updateData: any = {};
|
||||
|
||||
if (data.email) updateData.email = data.email;
|
||||
if (data.firstName !== undefined) updateData.firstName = data.firstName;
|
||||
if (data.lastName !== undefined) updateData.lastName = data.lastName;
|
||||
if (data.alias !== undefined) updateData.alias = data.alias;
|
||||
|
||||
// Hash password if provided
|
||||
if (data.password) {
|
||||
updateData.password = await bcrypt.hash(data.password, 10);
|
||||
}
|
||||
|
||||
const user = await User.query(knex).patchAndFetchById(id, updateData);
|
||||
return user;
|
||||
}
|
||||
|
||||
@Delete(':id')
|
||||
async deleteUser(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('id') id: string,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Delete user role assignments first
|
||||
await knex('user_roles').where({ userId: id }).delete();
|
||||
|
||||
// Delete the user
|
||||
await User.query(knex).deleteById(id);
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
@Post(':userId/roles')
|
||||
async addRoleToUser(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('userId') userId: string,
|
||||
@Body() data: { roleId: string },
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
// Check if assignment already exists
|
||||
const existing = await knex('user_roles')
|
||||
.where({ userId, roleId: data.roleId })
|
||||
.first();
|
||||
|
||||
if (existing) {
|
||||
return { success: true, message: 'Role already assigned' };
|
||||
}
|
||||
|
||||
await knex('user_roles').insert({
|
||||
id: knex.raw('(UUID())'),
|
||||
userId,
|
||||
roleId: data.roleId,
|
||||
created_at: knex.fn.now(),
|
||||
updated_at: knex.fn.now(),
|
||||
});
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
@Delete(':userId/roles/:roleId')
|
||||
async removeRoleFromUser(
|
||||
@TenantId() tenantId: string,
|
||||
@Param('userId') userId: string,
|
||||
@Param('roleId') roleId: string,
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
await knex('user_roles')
|
||||
.where({ userId, roleId })
|
||||
.delete();
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
}
|
||||
53
backend/src/saved-list-view/dto/saved-list-view.dto.ts
Normal file
53
backend/src/saved-list-view/dto/saved-list-view.dto.ts
Normal file
@@ -0,0 +1,53 @@
|
||||
import { IsString, IsNotEmpty, IsArray, IsOptional } from 'class-validator';
|
||||
|
||||
export class CreateSavedViewDto {
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
name: string;
|
||||
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
objectApiName: string;
|
||||
|
||||
@IsArray()
|
||||
filters: Array<{
|
||||
field: string;
|
||||
operator: string;
|
||||
value?: any;
|
||||
values?: any[];
|
||||
from?: string;
|
||||
to?: string;
|
||||
}>;
|
||||
|
||||
@IsOptional()
|
||||
sort?: { field: string; direction: 'asc' | 'desc' } | null;
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
description?: string;
|
||||
}
|
||||
|
||||
export class UpdateSavedViewDto {
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
@IsNotEmpty()
|
||||
name?: string;
|
||||
|
||||
@IsOptional()
|
||||
@IsArray()
|
||||
filters?: Array<{
|
||||
field: string;
|
||||
operator: string;
|
||||
value?: any;
|
||||
values?: any[];
|
||||
from?: string;
|
||||
to?: string;
|
||||
}>;
|
||||
|
||||
@IsOptional()
|
||||
sort?: { field: string; direction: 'asc' | 'desc' } | null;
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
description?: string;
|
||||
}
|
||||
92
backend/src/saved-list-view/saved-list-view.controller.ts
Normal file
92
backend/src/saved-list-view/saved-list-view.controller.ts
Normal file
@@ -0,0 +1,92 @@
|
||||
import {
|
||||
Controller,
|
||||
Get,
|
||||
Post,
|
||||
Patch,
|
||||
Delete,
|
||||
Body,
|
||||
Param,
|
||||
UseGuards,
|
||||
ForbiddenException,
|
||||
NotFoundException,
|
||||
} from '@nestjs/common';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { CurrentUser } from '../auth/current-user.decorator';
|
||||
import { TenantId } from '../tenant/tenant.decorator';
|
||||
import { SavedListViewService } from './saved-list-view.service';
|
||||
import { CreateSavedViewDto, UpdateSavedViewDto } from './dto/saved-list-view.dto';
|
||||
import { CreateRecordShareDto } from '../rbac/dto/create-record-share.dto';
|
||||
|
||||
@Controller('saved-views')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
export class SavedListViewController {
|
||||
constructor(private readonly savedListViewService: SavedListViewService) {}
|
||||
|
||||
@Get(':objectApiName')
|
||||
findByObject(
|
||||
@TenantId() tenantId: string,
|
||||
@CurrentUser() user: any,
|
||||
@Param('objectApiName') objectApiName: string,
|
||||
) {
|
||||
return this.savedListViewService.findByObject(tenantId, user.userId, objectApiName);
|
||||
}
|
||||
|
||||
@Post()
|
||||
create(
|
||||
@TenantId() tenantId: string,
|
||||
@CurrentUser() user: any,
|
||||
@Body() dto: CreateSavedViewDto,
|
||||
) {
|
||||
return this.savedListViewService.create(tenantId, user.userId, dto);
|
||||
}
|
||||
|
||||
@Patch(':id')
|
||||
update(
|
||||
@TenantId() tenantId: string,
|
||||
@CurrentUser() user: any,
|
||||
@Param('id') id: string,
|
||||
@Body() dto: UpdateSavedViewDto,
|
||||
) {
|
||||
return this.savedListViewService.update(tenantId, user.userId, id, dto);
|
||||
}
|
||||
|
||||
@Delete(':id')
|
||||
remove(
|
||||
@TenantId() tenantId: string,
|
||||
@CurrentUser() user: any,
|
||||
@Param('id') id: string,
|
||||
) {
|
||||
return this.savedListViewService.remove(tenantId, user.userId, id);
|
||||
}
|
||||
|
||||
// ── Sharing endpoints (reuse record_shares table) ────────────────────────
|
||||
|
||||
@Get(':id/shares')
|
||||
getShares(
|
||||
@TenantId() tenantId: string,
|
||||
@CurrentUser() user: any,
|
||||
@Param('id') id: string,
|
||||
) {
|
||||
return this.savedListViewService.getShares(tenantId, user.userId, id);
|
||||
}
|
||||
|
||||
@Post(':id/shares')
|
||||
createShare(
|
||||
@TenantId() tenantId: string,
|
||||
@CurrentUser() user: any,
|
||||
@Param('id') id: string,
|
||||
@Body() dto: CreateRecordShareDto,
|
||||
) {
|
||||
return this.savedListViewService.createShare(tenantId, user.userId, id, dto);
|
||||
}
|
||||
|
||||
@Delete(':id/shares/:shareId')
|
||||
removeShare(
|
||||
@TenantId() tenantId: string,
|
||||
@CurrentUser() user: any,
|
||||
@Param('id') id: string,
|
||||
@Param('shareId') shareId: string,
|
||||
) {
|
||||
return this.savedListViewService.removeShare(tenantId, user.userId, id, shareId);
|
||||
}
|
||||
}
|
||||
12
backend/src/saved-list-view/saved-list-view.module.ts
Normal file
12
backend/src/saved-list-view/saved-list-view.module.ts
Normal file
@@ -0,0 +1,12 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { SavedListViewService } from './saved-list-view.service';
|
||||
import { SavedListViewController } from './saved-list-view.controller';
|
||||
import { TenantModule } from '../tenant/tenant.module';
|
||||
|
||||
@Module({
|
||||
imports: [TenantModule],
|
||||
controllers: [SavedListViewController],
|
||||
providers: [SavedListViewService],
|
||||
exports: [SavedListViewService],
|
||||
})
|
||||
export class SavedListViewModule {}
|
||||
264
backend/src/saved-list-view/saved-list-view.service.ts
Normal file
264
backend/src/saved-list-view/saved-list-view.service.ts
Normal file
@@ -0,0 +1,264 @@
|
||||
import { Injectable, NotFoundException, ForbiddenException, BadRequestException } from '@nestjs/common';
|
||||
import { TenantDatabaseService } from '../tenant/tenant-database.service';
|
||||
import { CreateSavedViewDto, UpdateSavedViewDto } from './dto/saved-list-view.dto';
|
||||
import { RecordShare } from '../models/record-share.model';
|
||||
import { ObjectDefinition } from '../models/object-definition.model';
|
||||
|
||||
@Injectable()
|
||||
export class SavedListViewService {
|
||||
constructor(private readonly tenantDbService: TenantDatabaseService) {}
|
||||
|
||||
// ── Helpers ──────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Resolves the system object_definition ID for SavedListView.
|
||||
* This is needed to create record_shares rows for saved views.
|
||||
*/
|
||||
private async getSavedViewObjectDefId(knex: any): Promise<string> {
|
||||
const objectDef = await ObjectDefinition.query(knex)
|
||||
.findOne({ apiName: 'SavedListView' });
|
||||
if (!objectDef) {
|
||||
throw new BadRequestException(
|
||||
'SavedListView system object not found. Please run migrations.',
|
||||
);
|
||||
}
|
||||
return objectDef.id;
|
||||
}
|
||||
|
||||
// ── CRUD ─────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Returns all saved views visible to the user for a given object:
|
||||
* - Views owned by the user
|
||||
* - Views shared with the user via record_shares
|
||||
*/
|
||||
async findByObject(tenantId: string, userId: string, objectApiName: string) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const objectDefId = await this.getSavedViewObjectDefId(knex);
|
||||
|
||||
// IDs of views shared with this user via record_shares
|
||||
const sharedViewIds = await RecordShare.query(knex)
|
||||
.where({ objectDefinitionId: objectDefId, granteeUserId: userId })
|
||||
.whereNull('revokedAt')
|
||||
.where(builder => {
|
||||
builder.whereNull('expiresAt').orWhere('expiresAt', '>', new Date());
|
||||
})
|
||||
.select('recordId');
|
||||
|
||||
const sharedIds = sharedViewIds.map((s: any) => s.recordId);
|
||||
|
||||
const rows = await knex('saved_list_views')
|
||||
.where({ object_api_name: objectApiName })
|
||||
.andWhere(function () {
|
||||
this.where({ user_id: userId });
|
||||
if (sharedIds.length > 0) {
|
||||
this.orWhereIn('id', sharedIds);
|
||||
}
|
||||
})
|
||||
.orderBy('created_at', 'asc');
|
||||
|
||||
return rows.map((r: any) => this.deserialize(r, userId));
|
||||
}
|
||||
|
||||
async create(tenantId: string, userId: string, dto: CreateSavedViewDto) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const id = require('crypto').randomUUID();
|
||||
|
||||
await knex('saved_list_views').insert({
|
||||
id,
|
||||
name: dto.name,
|
||||
object_api_name: dto.objectApiName,
|
||||
user_id: userId,
|
||||
is_shared: false,
|
||||
strategy: 'query',
|
||||
filters: JSON.stringify(dto.filters || []),
|
||||
sort: dto.sort ? JSON.stringify(dto.sort) : null,
|
||||
description: dto.description || null,
|
||||
});
|
||||
|
||||
const row = await knex('saved_list_views').where({ id }).first();
|
||||
return this.deserialize(row, userId);
|
||||
}
|
||||
|
||||
async update(tenantId: string, userId: string, id: string, dto: UpdateSavedViewDto) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const existing = await knex('saved_list_views').where({ id }).first();
|
||||
if (!existing) throw new NotFoundException(`Saved view ${id} not found`);
|
||||
if (existing.user_id !== userId) {
|
||||
throw new ForbiddenException('You can only modify views you own');
|
||||
}
|
||||
|
||||
const updates: Record<string, any> = { updated_at: knex.fn.now() };
|
||||
if (dto.name !== undefined) updates.name = dto.name;
|
||||
if (dto.filters !== undefined) updates.filters = JSON.stringify(dto.filters);
|
||||
if (dto.sort !== undefined) updates.sort = dto.sort ? JSON.stringify(dto.sort) : null;
|
||||
if (dto.description !== undefined) updates.description = dto.description;
|
||||
|
||||
await knex('saved_list_views').where({ id }).update(updates);
|
||||
|
||||
const row = await knex('saved_list_views').where({ id }).first();
|
||||
return this.deserialize(row, userId);
|
||||
}
|
||||
|
||||
async remove(tenantId: string, userId: string, id: string) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const existing = await knex('saved_list_views').where({ id }).first();
|
||||
if (!existing) throw new NotFoundException(`Saved view ${id} not found`);
|
||||
if (existing.user_id !== userId) {
|
||||
throw new ForbiddenException('You can only delete views you own');
|
||||
}
|
||||
|
||||
// Also clean up any record_shares for this view
|
||||
const objectDefId = await this.getSavedViewObjectDefId(knex);
|
||||
await RecordShare.query(knex)
|
||||
.where({ objectDefinitionId: objectDefId, recordId: id })
|
||||
.delete();
|
||||
|
||||
await knex('saved_list_views').where({ id }).delete();
|
||||
return { deleted: true };
|
||||
}
|
||||
|
||||
// ── Sharing via record_shares ────────────────────────────────────────────
|
||||
|
||||
async getShares(tenantId: string, userId: string, viewId: string) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const view = await knex('saved_list_views').where({ id: viewId }).first();
|
||||
if (!view) throw new NotFoundException('Saved view not found');
|
||||
if (view.user_id !== userId) {
|
||||
throw new ForbiddenException('Only the view owner can manage sharing');
|
||||
}
|
||||
|
||||
const objectDefId = await this.getSavedViewObjectDefId(knex);
|
||||
|
||||
const shares = await RecordShare.query(knex)
|
||||
.where({ objectDefinitionId: objectDefId, recordId: viewId })
|
||||
.whereNull('revokedAt')
|
||||
.where(builder => {
|
||||
builder.whereNull('expiresAt').orWhere('expiresAt', '>', new Date());
|
||||
})
|
||||
.withGraphFetched('[granteeUser]')
|
||||
.orderBy('createdAt', 'desc');
|
||||
|
||||
return shares;
|
||||
}
|
||||
|
||||
async createShare(
|
||||
tenantId: string,
|
||||
userId: string,
|
||||
viewId: string,
|
||||
dto: { granteeUserId: string; canRead: boolean; canEdit: boolean; canDelete: boolean; expiresAt?: string },
|
||||
) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const view = await knex('saved_list_views').where({ id: viewId }).first();
|
||||
if (!view) throw new NotFoundException('Saved view not found');
|
||||
if (view.user_id !== userId) {
|
||||
throw new ForbiddenException('Only the view owner can share it');
|
||||
}
|
||||
if (dto.granteeUserId === userId) {
|
||||
throw new BadRequestException('Cannot share a view with yourself');
|
||||
}
|
||||
|
||||
const objectDefId = await this.getSavedViewObjectDefId(knex);
|
||||
|
||||
// Upsert: if non-revoked share already exists for this grantee, update it
|
||||
const existing = await RecordShare.query(knex)
|
||||
.where({
|
||||
objectDefinitionId: objectDefId,
|
||||
recordId: viewId,
|
||||
granteeUserId: dto.granteeUserId,
|
||||
})
|
||||
.whereNull('revokedAt')
|
||||
.first();
|
||||
|
||||
if (existing) {
|
||||
await RecordShare.query(knex)
|
||||
.patchAndFetchById(existing.id, {
|
||||
accessLevel: {
|
||||
canRead: dto.canRead,
|
||||
canEdit: dto.canEdit,
|
||||
canDelete: dto.canDelete,
|
||||
},
|
||||
expiresAt: dto.expiresAt
|
||||
? (knex.raw('?', [new Date(dto.expiresAt).toISOString().slice(0, 19).replace('T', ' ')]) as any)
|
||||
: null,
|
||||
} as any);
|
||||
|
||||
return RecordShare.query(knex)
|
||||
.findById(existing.id)
|
||||
.withGraphFetched('[granteeUser]');
|
||||
}
|
||||
|
||||
const share = await RecordShare.query(knex).insertAndFetch({
|
||||
objectDefinitionId: objectDefId,
|
||||
recordId: viewId,
|
||||
granteeUserId: dto.granteeUserId,
|
||||
grantedByUserId: userId,
|
||||
accessLevel: {
|
||||
canRead: dto.canRead,
|
||||
canEdit: dto.canEdit,
|
||||
canDelete: dto.canDelete,
|
||||
},
|
||||
expiresAt: dto.expiresAt
|
||||
? (knex.raw('?', [new Date(dto.expiresAt).toISOString().slice(0, 19).replace('T', ' ')]) as any)
|
||||
: null,
|
||||
} as any);
|
||||
|
||||
return RecordShare.query(knex)
|
||||
.findById(share.id)
|
||||
.withGraphFetched('[granteeUser]');
|
||||
}
|
||||
|
||||
async removeShare(tenantId: string, userId: string, viewId: string, shareId: string) {
|
||||
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
|
||||
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
|
||||
|
||||
const view = await knex('saved_list_views').where({ id: viewId }).first();
|
||||
if (!view) throw new NotFoundException('Saved view not found');
|
||||
if (view.user_id !== userId) {
|
||||
throw new ForbiddenException('Only the view owner can manage sharing');
|
||||
}
|
||||
|
||||
const share = await RecordShare.query(knex).findById(shareId);
|
||||
if (!share) throw new NotFoundException('Share not found');
|
||||
|
||||
// Soft-revoke
|
||||
await RecordShare.query(knex)
|
||||
.findById(shareId)
|
||||
.patch({ revokedAt: knex.fn.now() } as any);
|
||||
|
||||
return { revoked: true };
|
||||
}
|
||||
|
||||
// ── Serialisation ────────────────────────────────────────────────────────
|
||||
|
||||
private deserialize(row: any, currentUserId: string) {
|
||||
return {
|
||||
id: row.id,
|
||||
name: row.name,
|
||||
objectApiName: row.object_api_name,
|
||||
userId: row.user_id,
|
||||
isOwner: row.user_id === currentUserId,
|
||||
isShared: Boolean(row.is_shared),
|
||||
strategy: row.strategy,
|
||||
filters: typeof row.filters === 'string' ? JSON.parse(row.filters) : (row.filters ?? []),
|
||||
sort: row.sort
|
||||
? (typeof row.sort === 'string' ? JSON.parse(row.sort) : row.sort)
|
||||
: null,
|
||||
description: row.description,
|
||||
createdAt: row.created_at,
|
||||
updatedAt: row.updated_at,
|
||||
};
|
||||
}
|
||||
}
|
||||
8
backend/src/search/meilisearch.module.ts
Normal file
8
backend/src/search/meilisearch.module.ts
Normal file
@@ -0,0 +1,8 @@
|
||||
import { Module } from '@nestjs/common';
|
||||
import { MeilisearchService } from './meilisearch.service';
|
||||
|
||||
@Module({
|
||||
providers: [MeilisearchService],
|
||||
exports: [MeilisearchService],
|
||||
})
|
||||
export class MeilisearchModule {}
|
||||
483
backend/src/search/meilisearch.service.ts
Normal file
483
backend/src/search/meilisearch.service.ts
Normal file
@@ -0,0 +1,483 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
import * as http from 'http';
|
||||
import * as https from 'https';
|
||||
|
||||
type MeiliConfig = {
|
||||
host: string;
|
||||
apiKey?: string;
|
||||
indexPrefix: string;
|
||||
};
|
||||
|
||||
type HybridSearchOptions = {
|
||||
embedder: string;
|
||||
semanticRatio?: number;
|
||||
};
|
||||
|
||||
type OpenAiEmbedderConfig = {
|
||||
embedderName: string;
|
||||
apiKey: string;
|
||||
model: string;
|
||||
documentTemplate: string;
|
||||
};
|
||||
|
||||
@Injectable()
|
||||
export class MeilisearchService {
|
||||
private readonly logger = new Logger(MeilisearchService.name);
|
||||
private readonly embedderCache = new Map<string, string>();
|
||||
private vectorStoreEnabled = false;
|
||||
|
||||
isEnabled(): boolean {
|
||||
return Boolean(this.getConfig());
|
||||
}
|
||||
|
||||
async searchRecord(
|
||||
tenantId: string,
|
||||
objectApiName: string,
|
||||
query: string,
|
||||
displayField?: string,
|
||||
): Promise<{ id: string; hit: any } | null> {
|
||||
const config = this.getConfig();
|
||||
if (!config) return null;
|
||||
|
||||
const indexName = this.buildIndexName(config, tenantId, objectApiName);
|
||||
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/search`;
|
||||
|
||||
console.log('querying Meilisearch index:', { indexName, query, displayField });
|
||||
|
||||
try {
|
||||
const response = await this.requestJson('POST', url, {
|
||||
q: query,
|
||||
limit: 5,
|
||||
}, this.buildHeaders(config));
|
||||
|
||||
if (!this.isSuccessStatus(response.status)) {
|
||||
this.logger.warn(
|
||||
`Meilisearch query failed for index ${indexName}: ${response.status}`,
|
||||
);
|
||||
return null;
|
||||
}
|
||||
|
||||
const hits = Array.isArray(response.body?.hits) ? response.body.hits : [];
|
||||
if (hits.length === 0) return null;
|
||||
|
||||
if (displayField) {
|
||||
const loweredQuery = query.toLowerCase();
|
||||
const exactMatch = hits.find((hit: any) => {
|
||||
const value = hit?.[displayField];
|
||||
return value && String(value).toLowerCase() === loweredQuery;
|
||||
});
|
||||
if (exactMatch?.id) {
|
||||
return { id: exactMatch.id, hit: exactMatch };
|
||||
}
|
||||
}
|
||||
|
||||
const match = hits[0];
|
||||
if (match?.id) {
|
||||
return { id: match.id, hit: match };
|
||||
}
|
||||
} catch (error) {
|
||||
this.logger.warn(`Meilisearch lookup failed: ${error.message}`);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
async searchRecords(
|
||||
tenantId: string,
|
||||
objectApiName: string,
|
||||
query: string,
|
||||
options?: { limit?: number; offset?: number },
|
||||
): Promise<{ hits: any[]; total: number }> {
|
||||
const config = this.getConfig();
|
||||
if (!config) return { hits: [], total: 0 };
|
||||
|
||||
const indexName = this.buildIndexName(config, tenantId, objectApiName);
|
||||
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/search`;
|
||||
const limit = Number.isFinite(Number(options?.limit)) ? Number(options?.limit) : 20;
|
||||
const offset = Number.isFinite(Number(options?.offset)) ? Number(options?.offset) : 0;
|
||||
|
||||
try {
|
||||
const response = await this.requestJson('POST', url, {
|
||||
q: query,
|
||||
limit,
|
||||
offset,
|
||||
}, this.buildHeaders(config));
|
||||
|
||||
console.log('Meilisearch response body:', response.body);
|
||||
|
||||
if (!this.isSuccessStatus(response.status)) {
|
||||
this.logger.warn(
|
||||
`Meilisearch query failed for index ${indexName}: ${response.status}`,
|
||||
);
|
||||
return { hits: [], total: 0 };
|
||||
}
|
||||
|
||||
const hits = Array.isArray(response.body?.hits) ? response.body.hits : [];
|
||||
const total =
|
||||
response.body?.estimatedTotalHits ??
|
||||
response.body?.nbHits ??
|
||||
hits.length;
|
||||
return { hits, total };
|
||||
} catch (error) {
|
||||
this.logger.warn(`Meilisearch query failed: ${error.message}`);
|
||||
return { hits: [], total: 0 };
|
||||
}
|
||||
}
|
||||
|
||||
async upsertRecord(
|
||||
tenantId: string,
|
||||
objectApiName: string,
|
||||
record: Record<string, any>,
|
||||
fieldsToIndex: string[],
|
||||
): Promise<void> {
|
||||
const config = this.getConfig();
|
||||
if (!config || !record?.id) return;
|
||||
|
||||
const indexName = this.buildIndexName(config, tenantId, objectApiName);
|
||||
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/documents?primaryKey=id`;
|
||||
const document = this.pickRecordFields(record, fieldsToIndex);
|
||||
|
||||
try {
|
||||
const response = await this.requestJson('POST', url, [document], this.buildHeaders(config));
|
||||
if (!this.isSuccessStatus(response.status)) {
|
||||
this.logger.warn(
|
||||
`Meilisearch upsert failed for index ${indexName}: ${response.status}`,
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
this.logger.warn(`Meilisearch upsert failed: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async deleteRecord(
|
||||
tenantId: string,
|
||||
objectApiName: string,
|
||||
recordId: string,
|
||||
): Promise<void> {
|
||||
const config = this.getConfig();
|
||||
if (!config || !recordId) return;
|
||||
|
||||
const indexName = this.buildIndexName(config, tenantId, objectApiName);
|
||||
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/documents/${encodeURIComponent(recordId)}`;
|
||||
|
||||
try {
|
||||
const response = await this.requestJson('DELETE', url, undefined, this.buildHeaders(config));
|
||||
if (!this.isSuccessStatus(response.status)) {
|
||||
this.logger.warn(
|
||||
`Meilisearch delete failed for index ${indexName}: ${response.status}`,
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
this.logger.warn(`Meilisearch delete failed: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
buildSemanticChunkIndexName(tenantId: string): string {
|
||||
const config = this.getConfig();
|
||||
const prefix = config?.indexPrefix || 'tenant_';
|
||||
return `${prefix}${tenantId}_semantic_chunks`.toLowerCase();
|
||||
}
|
||||
|
||||
async upsertDocuments(indexName: string, documents: Record<string, any>[]): Promise<void> {
|
||||
const config = this.getConfig();
|
||||
if (!config || !Array.isArray(documents) || documents.length === 0) return;
|
||||
|
||||
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/documents?primaryKey=id`;
|
||||
try {
|
||||
const response = await this.requestJson('POST', url, documents, this.buildHeaders(config));
|
||||
if (!this.isSuccessStatus(response.status)) {
|
||||
this.logger.warn(`Meilisearch document upsert failed for index ${indexName}: ${response.status}`);
|
||||
return;
|
||||
}
|
||||
// Meilisearch indexes (and embeds) documents asynchronously. Wait for the task
|
||||
// to complete so callers can immediately search and see the new documents.
|
||||
const taskUid = response.body?.taskUid ?? response.body?.uid;
|
||||
if (Number.isFinite(Number(taskUid))) {
|
||||
const succeeded = await this.waitForTask(config, Number(taskUid), 30000);
|
||||
if (!succeeded) {
|
||||
this.logger.warn(`Meilisearch indexing task did not succeed within timeout: taskUid=${taskUid} index=${indexName}`);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
this.logger.warn(`Meilisearch document upsert failed: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async searchIndex(
|
||||
indexName: string,
|
||||
query: string,
|
||||
limit = 20,
|
||||
hybrid?: HybridSearchOptions,
|
||||
): Promise<{ hits: any[]; total: number }> {
|
||||
const config = this.getConfig();
|
||||
if (!config) return { hits: [], total: 0 };
|
||||
|
||||
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/search`;
|
||||
try {
|
||||
const response = await this.requestJson(
|
||||
'POST',
|
||||
url,
|
||||
{
|
||||
q: query,
|
||||
limit,
|
||||
showRankingScore: true,
|
||||
...(hybrid ? { hybrid, showRankingScoreDetails: true } : {}),
|
||||
},
|
||||
this.buildHeaders(config),
|
||||
);
|
||||
|
||||
if (!this.isSuccessStatus(response.status)) {
|
||||
this.logger.warn(
|
||||
`Meilisearch search failed for index ${indexName}: ${response.status}`,
|
||||
);
|
||||
this.logger.warn(
|
||||
`Meilisearch search payload: ${JSON.stringify({ q: query, limit, hybrid })}`,
|
||||
);
|
||||
this.logger.warn(
|
||||
`Meilisearch search error body: ${JSON.stringify(response.body)}`,
|
||||
);
|
||||
// If hybrid is invalid (embedder missing), retry once without hybrid
|
||||
if (hybrid && response.body?.code === 'invalid_embedder') {
|
||||
const fallback = await this.requestJson(
|
||||
'POST',
|
||||
url,
|
||||
{ q: query, limit },
|
||||
this.buildHeaders(config),
|
||||
);
|
||||
if (this.isSuccessStatus(fallback.status)) {
|
||||
const hits = Array.isArray(fallback.body?.hits) ? fallback.body.hits : [];
|
||||
const total =
|
||||
fallback.body?.estimatedTotalHits ?? fallback.body?.nbHits ?? hits.length;
|
||||
this.logger.warn(
|
||||
`Meilisearch hybrid failed; fell back to lexical search for index ${indexName}.`,
|
||||
);
|
||||
return { hits, total };
|
||||
}
|
||||
}
|
||||
return { hits: [], total: 0 };
|
||||
}
|
||||
|
||||
const hits = Array.isArray(response.body?.hits) ? response.body.hits : [];
|
||||
const total = response.body?.estimatedTotalHits ?? response.body?.nbHits ?? hits.length;
|
||||
return { hits, total };
|
||||
} catch (error) {
|
||||
this.logger.warn(`Meilisearch search failed: ${error.message}`);
|
||||
return { hits: [], total: 0 };
|
||||
}
|
||||
}
|
||||
|
||||
private getConfig(): MeiliConfig | null {
|
||||
const host = process.env.MEILI_HOST || process.env.MEILISEARCH_HOST;
|
||||
if (!host) return null;
|
||||
const trimmedHost = host.replace(/\/+$/, '');
|
||||
const apiKey = process.env.MEILI_API_KEY || process.env.MEILISEARCH_API_KEY;
|
||||
const indexPrefix = process.env.MEILI_INDEX_PREFIX || 'tenant_';
|
||||
return { host: trimmedHost, apiKey, indexPrefix };
|
||||
}
|
||||
|
||||
private buildIndexName(config: MeiliConfig, tenantId: string, objectApiName: string): string {
|
||||
return `${config.indexPrefix}${tenantId}_${objectApiName}`.toLowerCase();
|
||||
}
|
||||
|
||||
private buildHeaders(config: MeiliConfig): Record<string, string> {
|
||||
const headers: Record<string, string> = {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'application/json',
|
||||
};
|
||||
if (config.apiKey) {
|
||||
headers['X-Meili-API-Key'] = config.apiKey;
|
||||
headers.Authorization = `Bearer ${config.apiKey}`;
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
private pickRecordFields(record: Record<string, any>, fields: string[]): Record<string, any> {
|
||||
const document: Record<string, any> = { id: record.id };
|
||||
for (const field of fields) {
|
||||
if (record[field] !== undefined) {
|
||||
document[field] = record[field];
|
||||
}
|
||||
}
|
||||
return document;
|
||||
}
|
||||
|
||||
private isSuccessStatus(status: number): boolean {
|
||||
return status >= 200 && status < 300;
|
||||
}
|
||||
|
||||
private requestJson(
|
||||
method: 'POST' | 'DELETE' | 'PATCH' | 'GET',
|
||||
url: string,
|
||||
payload: any,
|
||||
headers: Record<string, string>,
|
||||
): Promise<{ status: number; body: any }> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const parsedUrl = new URL(url);
|
||||
const client = parsedUrl.protocol === 'https:' ? https : http;
|
||||
const request = client.request(
|
||||
{
|
||||
method,
|
||||
hostname: parsedUrl.hostname,
|
||||
port: parsedUrl.port,
|
||||
path: `${parsedUrl.pathname}${parsedUrl.search}`,
|
||||
headers,
|
||||
},
|
||||
(response) => {
|
||||
let data = '';
|
||||
response.on('data', (chunk) => {
|
||||
data += chunk;
|
||||
});
|
||||
response.on('end', () => {
|
||||
if (!data) {
|
||||
resolve({ status: response.statusCode || 0, body: null });
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const body = JSON.parse(data);
|
||||
resolve({ status: response.statusCode || 0, body });
|
||||
} catch (error) {
|
||||
reject(error);
|
||||
}
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
request.on('error', reject);
|
||||
if (payload !== undefined && method !== 'GET') {
|
||||
request.write(JSON.stringify(payload));
|
||||
}
|
||||
request.end();
|
||||
});
|
||||
}
|
||||
|
||||
private async enableVectorStore(): Promise<void> {
|
||||
// Temporarily disabled to avoid the overhead of checking on every save.
|
||||
// Re-enable by removing the early return below.
|
||||
return;
|
||||
if (this.vectorStoreEnabled) return; // eslint-disable-line no-unreachable
|
||||
const meiliConfig = this.getConfig();
|
||||
if (!meiliConfig) return;
|
||||
const url = `${meiliConfig.host}/experimental-features`;
|
||||
try {
|
||||
const response = await this.requestJson(
|
||||
'PATCH',
|
||||
url,
|
||||
{ vectorStore: true },
|
||||
this.buildHeaders(meiliConfig),
|
||||
);
|
||||
if (this.isSuccessStatus(response.status)) {
|
||||
this.vectorStoreEnabled = true;
|
||||
this.logger.log('Meilisearch vector store experimental feature enabled');
|
||||
} else {
|
||||
this.logger.warn(
|
||||
`Failed to enable Meilisearch vector store: ${response.status} ${JSON.stringify(response.body)}`,
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
this.logger.warn(`Failed to enable Meilisearch vector store: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async ensureOpenAiEmbedder(
|
||||
indexName: string,
|
||||
config: OpenAiEmbedderConfig,
|
||||
): Promise<boolean> {
|
||||
const meiliConfig = this.getConfig();
|
||||
if (!meiliConfig || !config?.apiKey) return false;
|
||||
|
||||
await this.enableVectorStore();
|
||||
|
||||
const signature = JSON.stringify({
|
||||
embedderName: config.embedderName,
|
||||
model: config.model,
|
||||
documentTemplate: config.documentTemplate,
|
||||
apiKey: config.apiKey,
|
||||
});
|
||||
const cacheKey = `${indexName}:${config.embedderName}`;
|
||||
if (this.embedderCache.get(cacheKey) === signature) {
|
||||
return true;
|
||||
}
|
||||
|
||||
const url = `${meiliConfig.host}/indexes/${encodeURIComponent(indexName)}/settings/embedders`;
|
||||
try {
|
||||
const response = await this.requestJson(
|
||||
'PATCH',
|
||||
url,
|
||||
{
|
||||
[config.embedderName]: {
|
||||
source: 'openAi',
|
||||
model: config.model,
|
||||
apiKey: config.apiKey,
|
||||
documentTemplate: config.documentTemplate,
|
||||
},
|
||||
},
|
||||
this.buildHeaders(meiliConfig),
|
||||
);
|
||||
if (!this.isSuccessStatus(response.status)) {
|
||||
this.logger.warn(
|
||||
`Meilisearch embedder update failed for index ${indexName}: ${response.status}`,
|
||||
);
|
||||
this.logger.warn(
|
||||
`Meilisearch embedder error body: ${JSON.stringify(response.body)}`,
|
||||
);
|
||||
return false;
|
||||
}
|
||||
const taskUid = response.body?.taskUid ?? response.body?.uid;
|
||||
if (Number.isFinite(Number(taskUid))) {
|
||||
const succeeded = await this.waitForTask(meiliConfig, Number(taskUid), 8000);
|
||||
if (!succeeded) {
|
||||
this.logger.warn(`Meilisearch embedder task did not succeed: ${taskUid}`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
const hasEmbedder = await this.hasEmbedder(meiliConfig, indexName, config.embedderName);
|
||||
if (!hasEmbedder) {
|
||||
this.logger.warn(`Meilisearch embedder missing after update: ${config.embedderName}`);
|
||||
return false;
|
||||
}
|
||||
this.embedderCache.set(cacheKey, signature);
|
||||
return true;
|
||||
} catch (error) {
|
||||
this.logger.warn(`Meilisearch embedder update failed: ${error.message}`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
private async waitForTask(
|
||||
config: MeiliConfig,
|
||||
taskUid: number,
|
||||
timeoutMs = 8000,
|
||||
): Promise<boolean> {
|
||||
const url = `${config.host}/tasks/${taskUid}`;
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeoutMs) {
|
||||
const response = await this.requestJson('GET', url, undefined, this.buildHeaders(config));
|
||||
if (!this.isSuccessStatus(response.status)) {
|
||||
return false;
|
||||
}
|
||||
const status = response.body?.status;
|
||||
if (status === 'succeeded') return true;
|
||||
if (status === 'failed' || status === 'canceled') {
|
||||
this.logger.warn(`Meilisearch task ${taskUid} failed: ${JSON.stringify(response.body?.error)}`);
|
||||
return false;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 300));
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private async hasEmbedder(
|
||||
config: MeiliConfig,
|
||||
indexName: string,
|
||||
embedderName: string,
|
||||
): Promise<boolean> {
|
||||
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/settings/embedders`;
|
||||
const response = await this.requestJson('GET', url, undefined, this.buildHeaders(config));
|
||||
if (!this.isSuccessStatus(response.status)) {
|
||||
return false;
|
||||
}
|
||||
const embedders = response.body || {};
|
||||
return Boolean(embedders && embedders[embedderName]);
|
||||
}
|
||||
}
|
||||
368
backend/src/tenant/central-admin.controller.ts
Normal file
368
backend/src/tenant/central-admin.controller.ts
Normal file
@@ -0,0 +1,368 @@
|
||||
import {
|
||||
Controller,
|
||||
Get,
|
||||
Post,
|
||||
Put,
|
||||
Delete,
|
||||
Body,
|
||||
Param,
|
||||
Query,
|
||||
UseGuards,
|
||||
UnauthorizedException,
|
||||
Req,
|
||||
} from '@nestjs/common';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { CentralTenant, CentralDomain, CentralUser } from '../models/central.model';
|
||||
import { getCentralKnex, initCentralModels } from './central-database.service';
|
||||
import { TenantProvisioningService } from './tenant-provisioning.service';
|
||||
import { TenantDatabaseService } from './tenant-database.service';
|
||||
import * as bcrypt from 'bcrypt';
|
||||
|
||||
/**
|
||||
* Controller for managing central database entities (tenants, domains, users)
|
||||
* Only accessible when logged in as central admin
|
||||
*/
|
||||
@Controller('central')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
export class CentralAdminController {
|
||||
constructor(
|
||||
private readonly provisioningService: TenantProvisioningService,
|
||||
private readonly tenantDbService: TenantDatabaseService,
|
||||
) {
|
||||
// Initialize central models on controller creation
|
||||
initCentralModels();
|
||||
}
|
||||
|
||||
private checkCentralAdmin(req: any) {
|
||||
const subdomain = req.raw?.subdomain;
|
||||
const centralSubdomains = (process.env.CENTRAL_SUBDOMAINS || 'central,admin').split(',');
|
||||
|
||||
if (!subdomain || !centralSubdomains.includes(subdomain)) {
|
||||
throw new UnauthorizedException('This endpoint is only accessible to central administrators');
|
||||
}
|
||||
}
|
||||
|
||||
// ==================== TENANTS ====================
|
||||
|
||||
@Get('tenants')
|
||||
async getTenants(@Req() req: any) {
|
||||
this.checkCentralAdmin(req);
|
||||
return CentralTenant.query().withGraphFetched('domains');
|
||||
}
|
||||
|
||||
@Get('tenants/:id')
|
||||
async getTenant(@Req() req: any, @Param('id') id: string) {
|
||||
this.checkCentralAdmin(req);
|
||||
return CentralTenant.query()
|
||||
.findById(id)
|
||||
.withGraphFetched('domains');
|
||||
}
|
||||
|
||||
@Post('tenants')
|
||||
async createTenant(
|
||||
@Req() req: any,
|
||||
@Body() data: {
|
||||
name: string;
|
||||
slug?: string;
|
||||
primaryDomain: string;
|
||||
dbHost?: string;
|
||||
dbPort?: number;
|
||||
},
|
||||
) {
|
||||
this.checkCentralAdmin(req);
|
||||
|
||||
// Use the provisioning service to create tenant with database and migrations
|
||||
const result = await this.provisioningService.provisionTenant({
|
||||
name: data.name,
|
||||
slug: data.slug || data.name.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, ''),
|
||||
primaryDomain: data.primaryDomain,
|
||||
dbHost: data.dbHost,
|
||||
dbPort: data.dbPort,
|
||||
});
|
||||
|
||||
// Return the created tenant
|
||||
return CentralTenant.query()
|
||||
.findById(result.tenantId)
|
||||
.withGraphFetched('domains');
|
||||
}
|
||||
|
||||
@Put('tenants/:id')
|
||||
async updateTenant(
|
||||
@Req() req: any,
|
||||
@Param('id') id: string,
|
||||
@Body() data: {
|
||||
name?: string;
|
||||
slug?: string;
|
||||
dbHost?: string;
|
||||
dbPort?: number;
|
||||
dbName?: string;
|
||||
dbUsername?: string;
|
||||
status?: string;
|
||||
},
|
||||
) {
|
||||
this.checkCentralAdmin(req);
|
||||
return CentralTenant.query()
|
||||
.patchAndFetchById(id, data);
|
||||
}
|
||||
|
||||
@Delete('tenants/:id')
|
||||
async deleteTenant(@Req() req: any, @Param('id') id: string) {
|
||||
this.checkCentralAdmin(req);
|
||||
await CentralTenant.query().deleteById(id);
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
// Get users for a specific tenant
|
||||
@Get('tenants/:id/users')
|
||||
async getTenantUsers(@Req() req: any, @Param('id') tenantId: string) {
|
||||
this.checkCentralAdmin(req);
|
||||
|
||||
try {
|
||||
// Get tenant to verify it exists
|
||||
const tenant = await CentralTenant.query().findById(tenantId);
|
||||
|
||||
if (!tenant) {
|
||||
throw new UnauthorizedException('Tenant not found');
|
||||
}
|
||||
|
||||
// Connect to tenant database using tenant ID directly
|
||||
const tenantKnex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
// Fetch users from tenant database
|
||||
const users = await tenantKnex('users').select('*');
|
||||
|
||||
// Remove password from response
|
||||
return users.map(({ password, ...user }) => user);
|
||||
} catch (error) {
|
||||
console.error('Error fetching tenant users:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// Create a user in a specific tenant
|
||||
@Post('tenants/:id/users')
|
||||
async createTenantUser(
|
||||
@Req() req: any,
|
||||
@Param('id') tenantId: string,
|
||||
@Body() data: {
|
||||
email: string;
|
||||
password: string;
|
||||
firstName?: string;
|
||||
lastName?: string;
|
||||
},
|
||||
) {
|
||||
this.checkCentralAdmin(req);
|
||||
|
||||
try {
|
||||
// Get tenant to verify it exists
|
||||
const tenant = await CentralTenant.query().findById(tenantId);
|
||||
|
||||
if (!tenant) {
|
||||
throw new UnauthorizedException('Tenant not found');
|
||||
}
|
||||
|
||||
// Connect to tenant database using tenant ID directly
|
||||
const tenantKnex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
// Hash password
|
||||
const hashedPassword = await bcrypt.hash(data.password, 10);
|
||||
|
||||
// Generate UUID for the new user
|
||||
const userId = require('crypto').randomUUID();
|
||||
|
||||
// Create user in tenant database
|
||||
await tenantKnex('users').insert({
|
||||
id: userId,
|
||||
email: data.email,
|
||||
password: hashedPassword,
|
||||
firstName: data.firstName || null,
|
||||
lastName: data.lastName || null,
|
||||
created_at: new Date(),
|
||||
updated_at: new Date(),
|
||||
});
|
||||
|
||||
// Fetch and return the created user
|
||||
const user = await tenantKnex('users').where('id', userId).first();
|
||||
|
||||
if (!user) {
|
||||
throw new Error('Failed to create user');
|
||||
}
|
||||
|
||||
const { password, ...userWithoutPassword } = user;
|
||||
|
||||
return userWithoutPassword;
|
||||
} catch (error) {
|
||||
console.error('Error creating tenant user:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
// ==================== DOMAINS ====================
|
||||
|
||||
@Get('domains')
|
||||
async getDomains(
|
||||
@Req() req: any,
|
||||
@Query('parentId') parentId?: string,
|
||||
@Query('tenantId') tenantId?: string,
|
||||
) {
|
||||
this.checkCentralAdmin(req);
|
||||
|
||||
let query = CentralDomain.query().withGraphFetched('tenant');
|
||||
|
||||
// Filter by parent/tenant ID if provided (for related lists)
|
||||
if (parentId || tenantId) {
|
||||
query = query.where('tenantId', parentId || tenantId);
|
||||
}
|
||||
|
||||
return query;
|
||||
}
|
||||
|
||||
@Get('domains/:id')
|
||||
async getDomain(@Req() req: any, @Param('id') id: string) {
|
||||
this.checkCentralAdmin(req);
|
||||
return CentralDomain.query()
|
||||
.findById(id)
|
||||
.withGraphFetched('tenant');
|
||||
}
|
||||
|
||||
@Post('domains')
|
||||
async createDomain(
|
||||
@Req() req: any,
|
||||
@Body() data: {
|
||||
domain: string;
|
||||
tenantId: string;
|
||||
isPrimary?: boolean;
|
||||
},
|
||||
) {
|
||||
this.checkCentralAdmin(req);
|
||||
return CentralDomain.query().insert({
|
||||
domain: data.domain,
|
||||
tenantId: data.tenantId,
|
||||
isPrimary: data.isPrimary || false,
|
||||
});
|
||||
}
|
||||
|
||||
@Put('domains/:id')
|
||||
async updateDomain(
|
||||
@Req() req: any,
|
||||
@Param('id') id: string,
|
||||
@Body() data: {
|
||||
domain?: string;
|
||||
tenantId?: string;
|
||||
isPrimary?: boolean;
|
||||
},
|
||||
) {
|
||||
this.checkCentralAdmin(req);
|
||||
return CentralDomain.query()
|
||||
.patchAndFetchById(id, data);
|
||||
}
|
||||
|
||||
@Delete('domains/:id')
|
||||
async deleteDomain(@Req() req: any, @Param('id') id: string) {
|
||||
this.checkCentralAdmin(req);
|
||||
|
||||
// Get domain info before deleting to invalidate cache
|
||||
const domain = await CentralDomain.query().findById(id);
|
||||
|
||||
// Delete the domain
|
||||
await CentralDomain.query().deleteById(id);
|
||||
|
||||
// Invalidate tenant connection cache for this domain
|
||||
if (domain) {
|
||||
this.tenantDbService.removeTenantConnection(domain.domain);
|
||||
}
|
||||
|
||||
return { success: true };
|
||||
}
|
||||
|
||||
// ==================== USERS (Central Admin Users) ====================
|
||||
|
||||
@Get('users')
|
||||
async getUsers(@Req() req: any) {
|
||||
this.checkCentralAdmin(req);
|
||||
const users = await CentralUser.query();
|
||||
// Remove password from response
|
||||
return users.map(({ password, ...user }) => user);
|
||||
}
|
||||
|
||||
@Get('users/:id')
|
||||
async getUser(@Req() req: any, @Param('id') id: string) {
|
||||
this.checkCentralAdmin(req);
|
||||
const user = await CentralUser.query().findById(id);
|
||||
|
||||
if (!user) {
|
||||
throw new UnauthorizedException('User not found');
|
||||
}
|
||||
|
||||
const { password, ...userWithoutPassword } = user;
|
||||
return userWithoutPassword;
|
||||
}
|
||||
|
||||
@Post('users')
|
||||
async createUser(
|
||||
@Req() req: any,
|
||||
@Body() data: {
|
||||
email: string;
|
||||
password: string;
|
||||
firstName?: string;
|
||||
lastName?: string;
|
||||
role?: string;
|
||||
isActive?: boolean;
|
||||
},
|
||||
) {
|
||||
this.checkCentralAdmin(req);
|
||||
|
||||
const hashedPassword = await bcrypt.hash(data.password, 10);
|
||||
|
||||
const user = await CentralUser.query().insert({
|
||||
email: data.email,
|
||||
password: hashedPassword,
|
||||
firstName: data.firstName || null,
|
||||
lastName: data.lastName || null,
|
||||
role: data.role || 'admin',
|
||||
isActive: data.isActive !== undefined ? data.isActive : true,
|
||||
});
|
||||
|
||||
const { password, ...userWithoutPassword } = user;
|
||||
return userWithoutPassword;
|
||||
}
|
||||
|
||||
@Put('users/:id')
|
||||
async updateUser(
|
||||
@Req() req: any,
|
||||
@Param('id') id: string,
|
||||
@Body() data: {
|
||||
email?: string;
|
||||
password?: string;
|
||||
firstName?: string;
|
||||
lastName?: string;
|
||||
role?: string;
|
||||
isActive?: boolean;
|
||||
},
|
||||
) {
|
||||
this.checkCentralAdmin(req);
|
||||
|
||||
const updateData: any = { ...data };
|
||||
|
||||
// Hash password if provided
|
||||
if (data.password) {
|
||||
updateData.password = await bcrypt.hash(data.password, 10);
|
||||
} else {
|
||||
// Remove password from update if not provided
|
||||
delete updateData.password;
|
||||
}
|
||||
|
||||
const user = await CentralUser.query()
|
||||
.patchAndFetchById(id, updateData);
|
||||
|
||||
const { password, ...userWithoutPassword } = user;
|
||||
return userWithoutPassword;
|
||||
}
|
||||
|
||||
@Delete('users/:id')
|
||||
async deleteUser(@Req() req: any, @Param('id') id: string) {
|
||||
this.checkCentralAdmin(req);
|
||||
await CentralUser.query().deleteById(id);
|
||||
return { success: true };
|
||||
}
|
||||
}
|
||||
43
backend/src/tenant/central-database.service.ts
Normal file
43
backend/src/tenant/central-database.service.ts
Normal file
@@ -0,0 +1,43 @@
|
||||
import Knex from 'knex';
|
||||
import { Model } from 'objection';
|
||||
import { CentralTenant, CentralDomain, CentralUser } from '../models/central.model';
|
||||
|
||||
let centralKnex: Knex.Knex | null = null;
|
||||
|
||||
/**
|
||||
* Get or create a Knex instance for the central database
|
||||
* This is used for Objection models that work with central entities
|
||||
*/
|
||||
export function getCentralKnex(): Knex.Knex {
|
||||
if (!centralKnex) {
|
||||
const centralDbUrl = process.env.CENTRAL_DATABASE_URL;
|
||||
|
||||
if (!centralDbUrl) {
|
||||
throw new Error('CENTRAL_DATABASE_URL environment variable is not set');
|
||||
}
|
||||
|
||||
centralKnex = Knex({
|
||||
client: 'mysql2',
|
||||
connection: centralDbUrl,
|
||||
pool: {
|
||||
min: 2,
|
||||
max: 10,
|
||||
},
|
||||
});
|
||||
|
||||
// Bind Objection models to this knex instance
|
||||
Model.knex(centralKnex);
|
||||
}
|
||||
|
||||
return centralKnex;
|
||||
}
|
||||
|
||||
/**
|
||||
* Initialize central models with the knex instance
|
||||
*/
|
||||
export function initCentralModels() {
|
||||
const knex = getCentralKnex();
|
||||
CentralTenant.knex(knex);
|
||||
CentralDomain.knex(knex);
|
||||
CentralUser.knex(knex);
|
||||
}
|
||||
@@ -8,32 +8,116 @@ export class TenantDatabaseService {
|
||||
private readonly logger = new Logger(TenantDatabaseService.name);
|
||||
private tenantConnections: Map<string, Knex> = new Map();
|
||||
|
||||
async getTenantKnex(tenantIdOrSlug: string): Promise<Knex> {
|
||||
if (this.tenantConnections.has(tenantIdOrSlug)) {
|
||||
return this.tenantConnections.get(tenantIdOrSlug);
|
||||
/**
|
||||
* Get tenant database connection by domain (for subdomain-based authentication)
|
||||
* This is used when users log in via tenant subdomains
|
||||
*/
|
||||
async getTenantKnexByDomain(domain: string): Promise<Knex> {
|
||||
const cacheKey = `domain:${domain}`;
|
||||
|
||||
// Check if we have a cached connection
|
||||
if (this.tenantConnections.has(cacheKey)) {
|
||||
// Validate the domain still exists before returning cached connection
|
||||
const centralPrisma = getCentralPrisma();
|
||||
|
||||
try {
|
||||
const domainRecord = await centralPrisma.domain.findUnique({
|
||||
where: { domain },
|
||||
});
|
||||
|
||||
// If domain no longer exists, remove cached connection
|
||||
if (!domainRecord) {
|
||||
this.logger.warn(`Domain ${domain} no longer exists, removing cached connection`);
|
||||
await this.disconnectTenant(cacheKey);
|
||||
throw new Error(`Domain ${domain} not found`);
|
||||
}
|
||||
} catch (error) {
|
||||
// If domain doesn't exist, remove from cache and re-throw
|
||||
if (error.message.includes('not found')) {
|
||||
throw error;
|
||||
}
|
||||
// For other errors, log but continue with cached connection
|
||||
this.logger.warn(`Error validating domain ${domain}:`, error.message);
|
||||
}
|
||||
|
||||
return this.tenantConnections.get(cacheKey);
|
||||
}
|
||||
|
||||
const centralPrisma = getCentralPrisma();
|
||||
|
||||
// Try to find tenant by ID first, then by slug
|
||||
let tenant = await centralPrisma.tenant.findUnique({
|
||||
where: { id: tenantIdOrSlug },
|
||||
// Find tenant by domain
|
||||
const domainRecord = await centralPrisma.domain.findUnique({
|
||||
where: { domain },
|
||||
include: { tenant: true },
|
||||
});
|
||||
|
||||
if (!tenant) {
|
||||
tenant = await centralPrisma.tenant.findUnique({
|
||||
where: { slug: tenantIdOrSlug },
|
||||
});
|
||||
if (!domainRecord) {
|
||||
throw new Error(`Domain ${domain} not found`);
|
||||
}
|
||||
|
||||
const tenant = domainRecord.tenant;
|
||||
this.logger.log(`Found tenant by domain: ${domain} -> ${tenant.name}`);
|
||||
|
||||
if (tenant.status !== 'active') {
|
||||
throw new Error(`Tenant ${tenant.name} is not active`);
|
||||
}
|
||||
|
||||
// Create connection and cache it
|
||||
const tenantKnex = await this.createTenantConnection(tenant);
|
||||
this.tenantConnections.set(cacheKey, tenantKnex);
|
||||
|
||||
return tenantKnex;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get tenant database connection by tenant ID (for central admin operations)
|
||||
* This is used when central admin needs to access tenant databases
|
||||
*/
|
||||
async getTenantKnexById(tenantId: string): Promise<Knex> {
|
||||
const cacheKey = `id:${tenantId}`;
|
||||
|
||||
// Check if we have a cached connection (no validation needed for ID-based lookups)
|
||||
if (this.tenantConnections.has(cacheKey)) {
|
||||
return this.tenantConnections.get(cacheKey);
|
||||
}
|
||||
|
||||
const centralPrisma = getCentralPrisma();
|
||||
|
||||
// Find tenant by ID
|
||||
const tenant = await centralPrisma.tenant.findUnique({
|
||||
where: { id: tenantId },
|
||||
});
|
||||
|
||||
if (!tenant) {
|
||||
throw new Error(`Tenant ${tenantIdOrSlug} not found`);
|
||||
throw new Error(`Tenant ${tenantId} not found`);
|
||||
}
|
||||
|
||||
if (tenant.status !== 'active') {
|
||||
throw new Error(`Tenant ${tenantIdOrSlug} is not active`);
|
||||
throw new Error(`Tenant ${tenant.name} is not active`);
|
||||
}
|
||||
|
||||
this.logger.log(`Connecting to tenant database by ID: ${tenant.name}`);
|
||||
|
||||
// Create connection and cache it
|
||||
const tenantKnex = await this.createTenantConnection(tenant);
|
||||
this.tenantConnections.set(cacheKey, tenantKnex);
|
||||
|
||||
return tenantKnex;
|
||||
}
|
||||
|
||||
/**
|
||||
* Legacy method - delegates to domain-based lookup
|
||||
* @deprecated Use getTenantKnexByDomain or getTenantKnexById instead
|
||||
*/
|
||||
async getTenantKnex(tenantIdOrSlug: string): Promise<Knex> {
|
||||
// Assume it's a domain if it contains a dot
|
||||
return this.getTenantKnexByDomain(tenantIdOrSlug);
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a new Knex connection to a tenant database
|
||||
*/
|
||||
private async createTenantConnection(tenant: any): Promise<Knex> {
|
||||
// Decrypt password
|
||||
const decryptedPassword = this.decryptPassword(tenant.dbPassword);
|
||||
|
||||
@@ -64,7 +148,6 @@ export class TenantDatabaseService {
|
||||
throw error;
|
||||
}
|
||||
|
||||
this.tenantConnections.set(tenantIdOrSlug, tenantKnex);
|
||||
return tenantKnex;
|
||||
}
|
||||
|
||||
@@ -86,6 +169,36 @@ export class TenantDatabaseService {
|
||||
return domainRecord.tenant;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve tenant by ID or slug
|
||||
* Tries ID first, then falls back to slug
|
||||
*/
|
||||
async resolveTenantId(idOrSlug: string): Promise<string> {
|
||||
const centralPrisma = getCentralPrisma();
|
||||
|
||||
// Try by ID first
|
||||
let tenant = await centralPrisma.tenant.findUnique({
|
||||
where: { id: idOrSlug },
|
||||
});
|
||||
|
||||
// If not found, try by slug
|
||||
if (!tenant) {
|
||||
tenant = await centralPrisma.tenant.findUnique({
|
||||
where: { slug: idOrSlug },
|
||||
});
|
||||
}
|
||||
|
||||
if (!tenant) {
|
||||
throw new Error(`Tenant ${idOrSlug} not found`);
|
||||
}
|
||||
|
||||
if (tenant.status !== 'active') {
|
||||
throw new Error(`Tenant ${tenant.name} is not active`);
|
||||
}
|
||||
|
||||
return tenant.id;
|
||||
}
|
||||
|
||||
async disconnectTenant(tenantId: string) {
|
||||
const connection = this.tenantConnections.get(tenantId);
|
||||
if (connection) {
|
||||
@@ -129,4 +242,26 @@ export class TenantDatabaseService {
|
||||
decrypted += decipher.final('utf8');
|
||||
return decrypted;
|
||||
}
|
||||
|
||||
/**
|
||||
* Encrypt integrations config JSON object
|
||||
* @param config - Plain object containing integration credentials
|
||||
* @returns Encrypted JSON string
|
||||
*/
|
||||
encryptIntegrationsConfig(config: any): string {
|
||||
if (!config) return null;
|
||||
const jsonString = JSON.stringify(config);
|
||||
return this.encryptPassword(jsonString);
|
||||
}
|
||||
|
||||
/**
|
||||
* Decrypt integrations config JSON string
|
||||
* @param encryptedConfig - Encrypted JSON string
|
||||
* @returns Plain object with integration credentials
|
||||
*/
|
||||
decryptIntegrationsConfig(encryptedConfig: string): any {
|
||||
if (!encryptedConfig) return null;
|
||||
const decrypted = this.decryptPassword(encryptedConfig);
|
||||
return JSON.parse(decrypted);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -176,7 +176,7 @@ export class TenantProvisioningService {
|
||||
* Seed default data for new tenant
|
||||
*/
|
||||
private async seedDefaultData(tenantId: string) {
|
||||
const tenantKnex = await this.tenantDbService.getTenantKnex(tenantId);
|
||||
const tenantKnex = await this.tenantDbService.getTenantKnexById(tenantId);
|
||||
|
||||
try {
|
||||
// Create default roles
|
||||
|
||||
170
backend/src/tenant/tenant.controller.ts
Normal file
170
backend/src/tenant/tenant.controller.ts
Normal file
@@ -0,0 +1,170 @@
|
||||
import {
|
||||
Controller,
|
||||
Get,
|
||||
Put,
|
||||
Body,
|
||||
UseGuards,
|
||||
Req,
|
||||
} from '@nestjs/common';
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||
import { TenantDatabaseService } from './tenant-database.service';
|
||||
import { getCentralPrisma } from '../prisma/central-prisma.service';
|
||||
import { TenantId } from './tenant.decorator';
|
||||
|
||||
@Controller('tenant')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
export class TenantController {
|
||||
constructor(private readonly tenantDbService: TenantDatabaseService) {}
|
||||
|
||||
/**
|
||||
* Helper to find tenant by ID or domain
|
||||
*/
|
||||
private async findTenant(identifier: string) {
|
||||
const centralPrisma = getCentralPrisma();
|
||||
|
||||
// Check if identifier is a CUID (tenant ID) or a domain
|
||||
const isCUID = /^c[a-z0-9]{24}$/i.test(identifier);
|
||||
|
||||
if (isCUID) {
|
||||
// Look up by tenant ID directly
|
||||
return centralPrisma.tenant.findUnique({
|
||||
where: { id: identifier },
|
||||
select: { id: true, integrationsConfig: true },
|
||||
});
|
||||
} else {
|
||||
// Look up by domain
|
||||
const domainRecord = await centralPrisma.domain.findUnique({
|
||||
where: { domain: identifier },
|
||||
include: { tenant: { select: { id: true, integrationsConfig: true } } },
|
||||
});
|
||||
return domainRecord?.tenant;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get integrations configuration for the current tenant
|
||||
*/
|
||||
@Get('integrations')
|
||||
async getIntegrationsConfig(@TenantId() tenantIdentifier: string) {
|
||||
const tenant = await this.findTenant(tenantIdentifier);
|
||||
|
||||
if (!tenant || !tenant.integrationsConfig) {
|
||||
return { data: null };
|
||||
}
|
||||
|
||||
// Decrypt the config
|
||||
const config = this.tenantDbService.decryptIntegrationsConfig(
|
||||
tenant.integrationsConfig as any,
|
||||
);
|
||||
|
||||
// Return config with sensitive fields masked
|
||||
const maskedConfig = this.maskSensitiveFields(config);
|
||||
|
||||
return { data: maskedConfig };
|
||||
}
|
||||
|
||||
/**
|
||||
* Update integrations configuration for the current tenant
|
||||
*/
|
||||
@Put('integrations')
|
||||
async updateIntegrationsConfig(
|
||||
@TenantId() tenantIdentifier: string,
|
||||
@Body() body: { integrationsConfig: any },
|
||||
) {
|
||||
const { integrationsConfig } = body;
|
||||
|
||||
if (!tenantIdentifier) {
|
||||
throw new Error('Tenant identifier is missing from request');
|
||||
}
|
||||
|
||||
const tenant = await this.findTenant(tenantIdentifier);
|
||||
|
||||
if (!tenant) {
|
||||
throw new Error(`Tenant with identifier ${tenantIdentifier} not found`);
|
||||
}
|
||||
|
||||
// Merge with existing config to preserve masked values
|
||||
let finalConfig = integrationsConfig;
|
||||
if (tenant.integrationsConfig) {
|
||||
const existingConfig = this.tenantDbService.decryptIntegrationsConfig(
|
||||
tenant.integrationsConfig as any,
|
||||
);
|
||||
|
||||
// Replace masked values with actual values from existing config
|
||||
finalConfig = this.unmaskConfig(integrationsConfig, existingConfig);
|
||||
}
|
||||
|
||||
// Encrypt the config
|
||||
const encryptedConfig = this.tenantDbService.encryptIntegrationsConfig(
|
||||
finalConfig,
|
||||
);
|
||||
|
||||
// Update in database
|
||||
const centralPrisma = getCentralPrisma();
|
||||
await centralPrisma.tenant.update({
|
||||
where: { id: tenant.id },
|
||||
data: {
|
||||
integrationsConfig: encryptedConfig as any,
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
success: true,
|
||||
message: 'Integrations configuration updated successfully',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Unmask config by replacing masked values with actual values from existing config
|
||||
*/
|
||||
private unmaskConfig(newConfig: any, existingConfig: any): any {
|
||||
const result = { ...newConfig };
|
||||
|
||||
// Unmask Twilio credentials
|
||||
if (result.twilio && existingConfig.twilio) {
|
||||
if (result.twilio.authToken === '••••••••' && existingConfig.twilio.authToken) {
|
||||
result.twilio.authToken = existingConfig.twilio.authToken;
|
||||
}
|
||||
if (result.twilio.apiSecret === '••••••••' && existingConfig.twilio.apiSecret) {
|
||||
result.twilio.apiSecret = existingConfig.twilio.apiSecret;
|
||||
}
|
||||
}
|
||||
|
||||
// Unmask OpenAI credentials
|
||||
if (result.openai && existingConfig.openai) {
|
||||
if (result.openai.apiKey === '••••••••' && existingConfig.openai.apiKey) {
|
||||
result.openai.apiKey = existingConfig.openai.apiKey;
|
||||
}
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mask sensitive fields for API responses
|
||||
*/
|
||||
private maskSensitiveFields(config: any): any {
|
||||
if (!config) return null;
|
||||
|
||||
const masked = { ...config };
|
||||
|
||||
// Mask Twilio credentials
|
||||
if (masked.twilio) {
|
||||
masked.twilio = {
|
||||
...masked.twilio,
|
||||
authToken: masked.twilio.authToken ? '••••••••' : '',
|
||||
apiSecret: masked.twilio.apiSecret ? '••••••••' : '',
|
||||
};
|
||||
}
|
||||
|
||||
// Mask OpenAI credentials
|
||||
if (masked.openai) {
|
||||
masked.openai = {
|
||||
...masked.openai,
|
||||
apiKey: masked.openai.apiKey ? '••••••••' : '',
|
||||
};
|
||||
}
|
||||
|
||||
return masked;
|
||||
}
|
||||
}
|
||||
@@ -14,20 +14,68 @@ export class TenantMiddleware implements NestMiddleware {
|
||||
next: () => void,
|
||||
) {
|
||||
try {
|
||||
// Extract subdomain from hostname
|
||||
const host = req.headers.host || '';
|
||||
const hostname = host.split(':')[0]; // Remove port if present
|
||||
const parts = hostname.split('.');
|
||||
|
||||
this.logger.log(`Host header: ${host}, hostname: ${hostname}, parts: ${JSON.stringify(parts)}`);
|
||||
|
||||
// For local development, accept x-tenant-id header
|
||||
// Priority 1: Check x-tenant-subdomain header from Nitro BFF proxy
|
||||
// This is the primary method when using the BFF architecture
|
||||
let subdomain = req.headers['x-tenant-subdomain'] as string | null;
|
||||
let tenantId = req.headers['x-tenant-id'] as string;
|
||||
let subdomain: string | null = null;
|
||||
|
||||
this.logger.log(`Host header: ${host}, hostname: ${hostname}, parts: ${JSON.stringify(parts)}, x-tenant-id: ${tenantId}`);
|
||||
if (subdomain) {
|
||||
this.logger.log(`Using x-tenant-subdomain header: ${subdomain}`);
|
||||
}
|
||||
|
||||
// If x-tenant-id is explicitly provided, use it directly
|
||||
// Priority 2: Fall back to extracting subdomain from Origin/Host headers
|
||||
// This supports direct backend access for development/testing
|
||||
if (!subdomain && !tenantId) {
|
||||
const host = req.headers.host || '';
|
||||
const hostname = host.split(':')[0];
|
||||
const origin = req.headers.origin as string;
|
||||
const referer = req.headers.referer as string;
|
||||
|
||||
let parts = hostname.split('.');
|
||||
|
||||
this.logger.log(`Host header: ${host}, hostname: ${hostname}, origin: ${origin}, referer: ${referer}`);
|
||||
|
||||
// Try to extract subdomain from Origin header first (for API calls from frontend)
|
||||
if (origin) {
|
||||
try {
|
||||
const originUrl = new URL(origin);
|
||||
const originHost = originUrl.hostname;
|
||||
parts = originHost.split('.');
|
||||
this.logger.log(`Using Origin header hostname: ${originHost}, parts: ${JSON.stringify(parts)}`);
|
||||
} catch (error) {
|
||||
this.logger.warn(`Failed to parse origin: ${origin}`);
|
||||
}
|
||||
} else if (referer) {
|
||||
// Fallback to Referer if no Origin
|
||||
try {
|
||||
const refererUrl = new URL(referer);
|
||||
const refererHost = refererUrl.hostname;
|
||||
parts = refererHost.split('.');
|
||||
this.logger.log(`Using Referer header hostname: ${refererHost}, parts: ${JSON.stringify(parts)}`);
|
||||
} catch (error) {
|
||||
this.logger.warn(`Failed to parse referer: ${referer}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Extract subdomain (e.g., "tenant1" from "tenant1.routebox.co")
|
||||
if (parts.length >= 3) {
|
||||
subdomain = parts[0];
|
||||
if (subdomain === 'www') {
|
||||
subdomain = null;
|
||||
}
|
||||
} else if (parts.length === 2 && parts[1] === 'localhost') {
|
||||
subdomain = parts[0];
|
||||
}
|
||||
}
|
||||
|
||||
this.logger.log(`Extracted subdomain: ${subdomain}, x-tenant-id: ${tenantId}`);
|
||||
|
||||
// Always attach subdomain to request if present
|
||||
if (subdomain) {
|
||||
(req as any).subdomain = subdomain;
|
||||
}
|
||||
|
||||
// If x-tenant-id is explicitly provided, use it directly but still keep subdomain
|
||||
if (tenantId) {
|
||||
this.logger.log(`Using explicit x-tenant-id: ${tenantId}`);
|
||||
(req as any).tenantId = tenantId;
|
||||
@@ -35,21 +83,22 @@ export class TenantMiddleware implements NestMiddleware {
|
||||
return;
|
||||
}
|
||||
|
||||
// Extract subdomain (e.g., "tenant1" from "tenant1.routebox.co")
|
||||
// For production domains with 3+ parts, extract first part as subdomain
|
||||
if (parts.length >= 3) {
|
||||
subdomain = parts[0];
|
||||
// Ignore www subdomain
|
||||
if (subdomain === 'www') {
|
||||
subdomain = null;
|
||||
}
|
||||
}
|
||||
// For development (e.g., tenant1.localhost), also check 2 parts
|
||||
else if (parts.length === 2 && parts[1] === 'localhost') {
|
||||
subdomain = parts[0];
|
||||
// Always attach subdomain to request if present
|
||||
if (subdomain) {
|
||||
(req as any).subdomain = subdomain;
|
||||
}
|
||||
|
||||
this.logger.log(`Extracted subdomain: ${subdomain}`);
|
||||
// Check if this is a central subdomain
|
||||
const centralSubdomains = (process.env.CENTRAL_SUBDOMAINS || 'central,admin').split(',');
|
||||
const isCentral = subdomain && centralSubdomains.includes(subdomain);
|
||||
|
||||
// If it's a central subdomain, skip tenant resolution
|
||||
if (isCentral) {
|
||||
this.logger.log(`Central subdomain detected: ${subdomain}, skipping tenant resolution`);
|
||||
(req as any).subdomain = subdomain;
|
||||
next();
|
||||
return;
|
||||
}
|
||||
|
||||
// Get tenant by subdomain if available
|
||||
if (subdomain) {
|
||||
@@ -72,11 +121,8 @@ export class TenantMiddleware implements NestMiddleware {
|
||||
if (tenantId) {
|
||||
// Attach tenant info to request object
|
||||
(req as any).tenantId = tenantId;
|
||||
if (subdomain) {
|
||||
(req as any).subdomain = subdomain;
|
||||
}
|
||||
} else {
|
||||
this.logger.warn(`No tenant identified from host: ${hostname}`);
|
||||
this.logger.warn(`No tenant identified from host: ${subdomain}`);
|
||||
}
|
||||
|
||||
next();
|
||||
|
||||
@@ -3,11 +3,13 @@ import { TenantMiddleware } from './tenant.middleware';
|
||||
import { TenantDatabaseService } from './tenant-database.service';
|
||||
import { TenantProvisioningService } from './tenant-provisioning.service';
|
||||
import { TenantProvisioningController } from './tenant-provisioning.controller';
|
||||
import { CentralAdminController } from './central-admin.controller';
|
||||
import { TenantController } from './tenant.controller';
|
||||
import { PrismaModule } from '../prisma/prisma.module';
|
||||
|
||||
@Module({
|
||||
imports: [PrismaModule],
|
||||
controllers: [TenantProvisioningController],
|
||||
controllers: [TenantProvisioningController, CentralAdminController, TenantController],
|
||||
providers: [
|
||||
TenantDatabaseService,
|
||||
TenantProvisioningService,
|
||||
|
||||
214
backend/src/voice/audio-converter.service.ts
Normal file
214
backend/src/voice/audio-converter.service.ts
Normal file
@@ -0,0 +1,214 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
/**
|
||||
* Audio format converter for Twilio <-> OpenAI audio streaming
|
||||
*
|
||||
* Twilio Media Streams format:
|
||||
* - Codec: μ-law (G.711)
|
||||
* - Sample rate: 8kHz
|
||||
* - Encoding: base64
|
||||
* - Chunk size: 20ms (160 bytes)
|
||||
*
|
||||
* OpenAI Realtime API format:
|
||||
* - Codec: PCM16
|
||||
* - Sample rate: 24kHz
|
||||
* - Encoding: base64
|
||||
* - Mono channel
|
||||
*/
|
||||
@Injectable()
|
||||
export class AudioConverterService {
|
||||
private readonly logger = new Logger(AudioConverterService.name);
|
||||
|
||||
// μ-law decode lookup table
|
||||
private readonly MULAW_DECODE_TABLE = this.buildMuLawDecodeTable();
|
||||
|
||||
// μ-law encode lookup table
|
||||
private readonly MULAW_ENCODE_TABLE = this.buildMuLawEncodeTable();
|
||||
|
||||
/**
|
||||
* Build μ-law to linear PCM16 decode table
|
||||
*/
|
||||
private buildMuLawDecodeTable(): Int16Array {
|
||||
const table = new Int16Array(256);
|
||||
for (let i = 0; i < 256; i++) {
|
||||
const mulaw = ~i;
|
||||
const exponent = (mulaw >> 4) & 0x07;
|
||||
const mantissa = mulaw & 0x0f;
|
||||
let sample = (mantissa << 3) + 0x84;
|
||||
sample <<= exponent;
|
||||
sample -= 0x84;
|
||||
if ((mulaw & 0x80) === 0) {
|
||||
sample = -sample;
|
||||
}
|
||||
table[i] = sample;
|
||||
}
|
||||
return table;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build linear PCM16 to μ-law encode table
|
||||
*/
|
||||
private buildMuLawEncodeTable(): Uint8Array {
|
||||
const table = new Uint8Array(65536);
|
||||
for (let i = 0; i < 65536; i++) {
|
||||
const sample = (i - 32768);
|
||||
const sign = sample < 0 ? 0x80 : 0x00;
|
||||
const magnitude = Math.abs(sample);
|
||||
|
||||
// Add bias
|
||||
let biased = magnitude + 0x84;
|
||||
|
||||
// Find exponent
|
||||
let exponent = 7;
|
||||
for (let exp = 0; exp < 8; exp++) {
|
||||
if (biased <= (0xff << exp)) {
|
||||
exponent = exp;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Extract mantissa
|
||||
const mantissa = (biased >> (exponent + 3)) & 0x0f;
|
||||
|
||||
// Combine sign, exponent, mantissa
|
||||
const mulaw = ~(sign | (exponent << 4) | mantissa);
|
||||
table[i] = mulaw & 0xff;
|
||||
}
|
||||
return table;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decode μ-law audio to linear PCM16
|
||||
* @param mulawData - Buffer containing μ-law encoded audio
|
||||
* @returns Buffer containing PCM16 audio (16-bit little-endian)
|
||||
*/
|
||||
decodeMuLaw(mulawData: Buffer): Buffer {
|
||||
const pcm16 = Buffer.allocUnsafe(mulawData.length * 2);
|
||||
|
||||
for (let i = 0; i < mulawData.length; i++) {
|
||||
const sample = this.MULAW_DECODE_TABLE[mulawData[i]];
|
||||
pcm16.writeInt16LE(sample, i * 2);
|
||||
}
|
||||
|
||||
return pcm16;
|
||||
}
|
||||
|
||||
/**
|
||||
* Encode linear PCM16 to μ-law
|
||||
* @param pcm16Data - Buffer containing PCM16 audio (16-bit little-endian)
|
||||
* @returns Buffer containing μ-law encoded audio
|
||||
*/
|
||||
encodeMuLaw(pcm16Data: Buffer): Buffer {
|
||||
const mulaw = Buffer.allocUnsafe(pcm16Data.length / 2);
|
||||
|
||||
for (let i = 0; i < pcm16Data.length; i += 2) {
|
||||
const sample = pcm16Data.readInt16LE(i);
|
||||
const index = (sample + 32768) & 0xffff;
|
||||
mulaw[i / 2] = this.MULAW_ENCODE_TABLE[index];
|
||||
}
|
||||
|
||||
return mulaw;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resample audio from 8kHz to 24kHz (linear interpolation)
|
||||
* @param pcm16Data - Buffer containing 8kHz PCM16 audio
|
||||
* @returns Buffer containing 24kHz PCM16 audio
|
||||
*/
|
||||
resample8kTo24k(pcm16Data: Buffer): Buffer {
|
||||
const inputSamples = pcm16Data.length / 2;
|
||||
const outputSamples = Math.floor(inputSamples * 3); // 8k * 3 = 24k
|
||||
const output = Buffer.allocUnsafe(outputSamples * 2);
|
||||
|
||||
for (let i = 0; i < outputSamples; i++) {
|
||||
const srcIndex = i / 3;
|
||||
const srcIndexFloor = Math.floor(srcIndex);
|
||||
const srcIndexCeil = Math.min(srcIndexFloor + 1, inputSamples - 1);
|
||||
const fraction = srcIndex - srcIndexFloor;
|
||||
|
||||
const sample1 = pcm16Data.readInt16LE(srcIndexFloor * 2);
|
||||
const sample2 = pcm16Data.readInt16LE(srcIndexCeil * 2);
|
||||
|
||||
// Linear interpolation
|
||||
const interpolated = Math.round(sample1 + (sample2 - sample1) * fraction);
|
||||
output.writeInt16LE(interpolated, i * 2);
|
||||
}
|
||||
|
||||
return output;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resample audio from 24kHz to 8kHz (decimation with averaging)
|
||||
* @param pcm16Data - Buffer containing 24kHz PCM16 audio
|
||||
* @returns Buffer containing 8kHz PCM16 audio
|
||||
*/
|
||||
resample24kTo8k(pcm16Data: Buffer): Buffer {
|
||||
const inputSamples = pcm16Data.length / 2;
|
||||
const outputSamples = Math.floor(inputSamples / 3); // 24k / 3 = 8k
|
||||
const output = Buffer.allocUnsafe(outputSamples * 2);
|
||||
|
||||
for (let i = 0; i < outputSamples; i++) {
|
||||
// Average 3 samples for anti-aliasing
|
||||
const idx1 = Math.min(i * 3, inputSamples - 1);
|
||||
const idx2 = Math.min(i * 3 + 1, inputSamples - 1);
|
||||
const idx3 = Math.min(i * 3 + 2, inputSamples - 1);
|
||||
|
||||
const sample1 = pcm16Data.readInt16LE(idx1 * 2);
|
||||
const sample2 = pcm16Data.readInt16LE(idx2 * 2);
|
||||
const sample3 = pcm16Data.readInt16LE(idx3 * 2);
|
||||
|
||||
const averaged = Math.round((sample1 + sample2 + sample3) / 3);
|
||||
output.writeInt16LE(averaged, i * 2);
|
||||
}
|
||||
|
||||
return output;
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert Twilio μ-law 8kHz to OpenAI PCM16 24kHz
|
||||
* @param twilioBase64 - Base64-encoded μ-law audio from Twilio
|
||||
* @returns Base64-encoded PCM16 24kHz audio for OpenAI
|
||||
*/
|
||||
twilioToOpenAI(twilioBase64: string): string {
|
||||
try {
|
||||
// Decode base64
|
||||
const mulawBuffer = Buffer.from(twilioBase64, 'base64');
|
||||
|
||||
// μ-law -> PCM16
|
||||
const pcm16_8k = this.decodeMuLaw(mulawBuffer);
|
||||
|
||||
// 8kHz -> 24kHz
|
||||
const pcm16_24k = this.resample8kTo24k(pcm16_8k);
|
||||
|
||||
// Encode to base64
|
||||
return pcm16_24k.toString('base64');
|
||||
} catch (error) {
|
||||
this.logger.error('Error converting Twilio to OpenAI audio', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert OpenAI PCM16 24kHz to Twilio μ-law 8kHz
|
||||
* @param openaiBase64 - Base64-encoded PCM16 24kHz audio from OpenAI
|
||||
* @returns Base64-encoded μ-law 8kHz audio for Twilio
|
||||
*/
|
||||
openAIToTwilio(openaiBase64: string): string {
|
||||
try {
|
||||
// Decode base64
|
||||
const pcm16_24k = Buffer.from(openaiBase64, 'base64');
|
||||
|
||||
// 24kHz -> 8kHz
|
||||
const pcm16_8k = this.resample24kTo8k(pcm16_24k);
|
||||
|
||||
// PCM16 -> μ-law
|
||||
const mulawBuffer = this.encodeMuLaw(pcm16_8k);
|
||||
|
||||
// Encode to base64
|
||||
return mulawBuffer.toString('base64');
|
||||
} catch (error) {
|
||||
this.logger.error('Error converting OpenAI to Twilio audio', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
25
backend/src/voice/dto/call-event.dto.ts
Normal file
25
backend/src/voice/dto/call-event.dto.ts
Normal file
@@ -0,0 +1,25 @@
|
||||
export interface CallEventDto {
|
||||
callSid: string;
|
||||
direction: 'inbound' | 'outbound';
|
||||
fromNumber: string;
|
||||
toNumber: string;
|
||||
status: string;
|
||||
}
|
||||
|
||||
export interface DtmfEventDto {
|
||||
callSid: string;
|
||||
digit: string;
|
||||
}
|
||||
|
||||
export interface TranscriptEventDto {
|
||||
callSid: string;
|
||||
transcript: string;
|
||||
isFinal: boolean;
|
||||
}
|
||||
|
||||
export interface AiSuggestionDto {
|
||||
callSid: string;
|
||||
suggestion: string;
|
||||
type: 'response' | 'action' | 'insight';
|
||||
data?: any;
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user