Compare commits

..

18 Commits

Author SHA1 Message Date
Francisco Gaona
c21274c86f WIP - record sharing 2025-12-28 21:31:02 +01:00
Francisco Gaona
358a216015 WIP - fix check for default writable 2025-12-28 20:39:44 +01:00
Francisco Gaona
868ce596ba WIP - remove console logs 2025-12-28 07:20:58 +01:00
Francisco Gaona
9aeb6d2b6f WIP - more fix for permissions 2025-12-28 07:18:47 +01:00
Francisco Gaona
ac4a4b68cd WIP - more progress with permissions 2025-12-28 06:48:03 +01:00
Francisco Gaona
88f656c3f5 WIP - permissions 2025-12-28 05:43:03 +01:00
Francisco Gaona
f4143ab106 WIP - Fix objection and model registry 2025-12-27 06:08:25 +01:00
Francisco Gaona
516e132611 WIP - move docs 2025-12-24 21:46:05 +01:00
Francisco Gaona
c5305490c1 WIP - use objection and working lookup field to owner 2025-12-24 21:43:58 +01:00
Francisco Gaona
4520f94b69 WIP - using objection base model to handle objects operations 2025-12-24 20:18:43 +01:00
Francisco Gaona
e4f1ba96ad WIP - custom migrations when object is created 2025-12-24 19:54:13 +01:00
Francisco Gaona
52c0849de2 WIP - manage tenant users from central 2025-12-24 12:17:22 +01:00
Francisco Gaona
b9fa3bd008 WIP - improve login to tenants by domains 2025-12-24 11:42:44 +01:00
Francisco Gaona
2bc672e4c5 WIP - some fixes 2025-12-24 10:54:19 +01:00
Francisco Gaona
962c84e6d2 WIP - fix lookup field 2025-12-24 00:05:15 +01:00
Francisco Gaona
fc1bec4de7 WIP - related lists and look up field 2025-12-23 23:59:04 +01:00
Francisco Gaona
0275b96014 WIP - central operations 2025-12-23 23:38:45 +01:00
Francisco Gaona
e4f3bad971 WIp - fix login into central 2025-12-23 22:16:58 +01:00
186 changed files with 4568 additions and 24180 deletions

View File

@@ -5,11 +5,6 @@ DATABASE_URL="mysql://platform:platform@db:3306/platform"
CENTRAL_DATABASE_URL="mysql://root:asjdnfqTash37faggT@db:3306/central_platform"
REDIS_URL="redis://redis:6379"
# Meilisearch (optional)
MEILI_HOST="http://meilisearch:7700"
MEILI_API_KEY="dev-meili-master-key"
MEILI_INDEX_PREFIX="tenant_"
# JWT, multi-tenant hints, etc.
JWT_SECRET="devsecret"
TENANCY_STRATEGY="single-db"

View File

@@ -1,5 +1,5 @@
NUXT_PORT=3001
NUXT_HOST=0.0.0.0
# Nitro BFF backend URL (server-only, not exposed to client)
BACKEND_URL=https://backend.routebox.co
# Point Nuxt to the API container (not localhost)
NUXT_PUBLIC_API_BASE_URL=http://jupiter.routebox.co:3000

View File

@@ -1,83 +0,0 @@
# Debugging Incoming Call Issue
## Current Problem
- Hear "Connecting to your call" message (TwiML is executing)
- No ring on mobile after "Connecting" message
- Click Accept button does nothing
- Call never connects
## Root Cause Hypothesis
The Twilio Device SDK is likely **NOT receiving the incoming call event** from Twilio's Signaling Server. This could be because:
1. **Identity Mismatch**: The Device's identity (from JWT token) doesn't match the `<Client>ID</Client>` in TwiML
2. **Device Not Registered**: Device registration isn't completing before the call arrives
3. **Twilio Signaling Issue**: Device isn't connected to Twilio Signaling Server
## How to Debug
### Step 1: Check Device Identity in Console
When you open the softphone dialog, **open Browser DevTools Console (F12)**
You should see logs like:
```
Token received, creating Device...
Token identity: e6d45fa3-a108-4085-81e5-a8e05e85e6fb
Token grants: {voice: {...}}
Registering Twilio Device...
✓ Twilio Device registered - ready to receive calls
Device identity: e6d45fa3-a108-4085-81e5-a8e05e85e6fb
Device state: ready
```
**Note the Device identity value** - e.g., "e6d45fa3-a108-4085-81e5-a8e05e85e6fb"
### Step 2: Check Backend Logs
When you make an inbound call, look for backend logs showing:
```
╔════════════════════════════════════════╗
║ === INBOUND CALL RECEIVED ===
╚════════════════════════════════════════╝
...
Client IDs to dial: e6d45fa3-a108-4085-81e5-a8e05e85e6fb
First Client ID format check: "e6d45fa3-a108-4085-81e5-a8e05e85e6fb" (length: 36)
```
### Step 3: Compare Identities
The Device identity from frontend console MUST MATCH the Client ID from backend logs.
**If they match**: The issue is with Twilio Signaling or Device SDK configuration
**If they don't match**: We found the bug - identity mismatch
### Step 4: Monitor Incoming Event
When you make the inbound call, keep watching the browser console for:
```
🔔 Twilio Device INCOMING event received: {...}
```
**If this appears**: The Device SDK IS receiving the call, so the Accept button issue is frontend
**If this doesn't appear**: The Device SDK is NOT receiving the call, so it's an identity/registration issue
## What Changed
- Frontend now relies on **Twilio Device SDK `incoming` event** (not Socket.IO) for showing incoming call
- Added comprehensive logging to Device initialization
- Added logging to Accept button handler
- Backend logs Device ID format for comparison
## Next Steps
1. Make an inbound call
2. Check browser console for the 5 logs above
3. Check backend logs for Client ID
4. Look for "🔔 Twilio Device INCOMING event" in browser console
5. Try clicking Accept and watch console for "📞 Accepting call" logs
6. Report back with:
- Device identity from console
- Client ID from backend logs
- Whether "🔔 Twilio Device INCOMING event" appears
- Whether any accept logs appear
## Important Files
- Backend: `/backend/src/voice/voice.controller.ts` (lines 205-210 show Client ID logging)
- Frontend: `/frontend/composables/useSoftphone.ts` (Device initialization and incoming handler)

251
IMPLEMENTATION_SUMMARY.md Normal file
View File

@@ -0,0 +1,251 @@
# Authorization System Implementation Summary
## ✅ Implementation Complete
A comprehensive polymorphic record sharing and authorization system has been implemented with CASL, Objection.js, and NestJS.
## What Was Built
### Backend (NestJS + Objection.js + CASL)
#### 1. Database Layer
- ✅ Migration for authorization tables (`20250128000001_add_authorization_system.js`)
- ✅ Updated Prisma schema with new models
- ✅ Objection.js models: `ObjectField`, `RoleRule`, `RecordShare`
- ✅ Updated existing models with new relations
#### 2. Authorization Core
-`AbilityFactory` - Builds CASL abilities from 3 layers (global, role, share)
- ✅ Query scoping utilities for SQL-level authorization
- ✅ Guards and decorators (`AbilitiesGuard`, `@CheckAbility()`, `@CurrentUser()`)
- ✅ Middleware for attaching abilities to requests
#### 3. API Endpoints
-**ShareController** - CRUD for record shares
- POST /shares - Create share
- GET /shares/record/:objectDefinitionId/:recordId - List shares
- GET /shares/granted - Shares granted by user
- GET /shares/received - Shares received by user
- PATCH /shares/:id - Update share
- DELETE /shares/:id - Revoke share
-**RoleController** - Role management
- Standard CRUD for roles
- RoleRuleController for CASL rules
-**ObjectAccessController** - Object-level permissions
- GET/PUT /setup/objects/:apiName/access
- POST /setup/objects/:apiName/fields/:fieldKey/permissions
- PUT /setup/objects/:apiName/field-permissions
### Frontend (Nuxt 3 + Vue 3)
#### 4. Object Management Enhancement
- ✅ Added "Access & Permissions" tab to object setup page
-`ObjectAccessSettings.vue` component:
- Configure access model (public/owner/mixed)
- Set public CRUD permissions
- Configure owner field
- Set field-level read/write permissions
#### 5. Role Management
- ✅ New page: `/setup/roles`
-`RolePermissionsEditor.vue` component:
- Configure CRUD permissions per object
- Apply conditions (e.g., own records only)
- Visual permission matrix
#### 6. Record Sharing
-`RecordShareDialog.vue` component:
- List current shares
- Add new shares with permissions
- Field-level scoping
- Expiration dates
- Revoke shares
## Key Features
### 🌍 Global Object Policies
- Public/private access models
- Default CRUD permissions per object
- Configurable owner field
- Field-level default permissions
### 👥 Role-Based Access
- CASL rules stored in database
- Per-object permissions
- Condition-based rules (e.g., ownerId matching)
- Multiple actions per rule
### 🔗 Per-Record Sharing
- Polymorphic design (works with any object type)
- Grant read/update access to specific users
- Optional field-level scoping
- Expiration and revocation support
- Track who granted each share
### 🔒 SQL Query Scoping
- Critical for list endpoints
- Ensures users only see authorized records
- Combines ownership + sharing logic
- Works with public access flags
## File Structure
```
backend/
├── migrations/tenant/
│ └── 20250128000001_add_authorization_system.js
├── src/
│ ├── auth/
│ │ ├── ability.factory.ts (CASL ability builder)
│ │ ├── query-scope.util.ts (SQL scoping utilities)
│ │ ├── guards/
│ │ │ └── abilities.guard.ts
│ │ ├── decorators/
│ │ │ ├── auth.decorators.ts
│ │ │ └── check-ability.decorator.ts
│ │ └── middleware/
│ │ └── ability.middleware.ts
│ ├── models/
│ │ ├── object-field.model.ts
│ │ ├── role-rule.model.ts
│ │ └── record-share.model.ts
│ ├── rbac/
│ │ ├── share.controller.ts
│ │ └── role.controller.ts
│ └── object/
│ └── object-access.controller.ts
frontend/
├── components/
│ ├── ObjectAccessSettings.vue
│ ├── RecordShareDialog.vue
│ └── RolePermissionsEditor.vue
└── pages/
├── setup/
│ ├── objects/[apiName].vue (enhanced with access tab)
│ └── roles.vue
└── ...
docs/
└── AUTHORIZATION_SYSTEM.md (comprehensive documentation)
```
## Next Steps
### 1. Run the Migration
```bash
cd backend
npm run migrate:latest
```
### 2. Initialize Existing Objects
Set default access models for existing object definitions:
```sql
UPDATE object_definitions
SET
access_model = 'owner',
public_read = false,
public_create = false,
public_update = false,
public_delete = false,
owner_field = 'ownerId'
WHERE access_model IS NULL;
```
### 3. Apply Query Scoping
Update existing controllers to use query scoping:
```typescript
import { applyReadScope } from '@/auth/query-scope.util';
// In your list endpoint
async findAll(@CurrentUser() user: User) {
const objectDef = await ObjectDefinition.query(this.knex)
.findOne({ apiName: 'YourObject' });
let query = YourModel.query(this.knex);
query = applyReadScope(query, user, objectDef, this.knex);
return query;
}
```
### 4. Add Route Protection
Use guards on sensitive endpoints:
```typescript
@UseGuards(JwtAuthGuard, AbilitiesGuard)
@CheckAbility({ action: 'update', subject: 'Post' })
async update(@Body() data: any) {
// Only users with 'update' permission on 'Post' can access
}
```
### 5. Frontend Integration
Add sharing button to record detail pages:
```vue
<template>
<div>
<!-- Your record details -->
<Button @click="showShareDialog = true">
<Share class="w-4 h-4 mr-2" />
Share
</Button>
<RecordShareDialog
:open="showShareDialog"
:object-definition-id="objectDefinition.id"
:record-id="record.id"
:fields="objectDefinition.fields"
@close="showShareDialog = false"
/>
</div>
</template>
```
## Testing Checklist
- [ ] Run database migration successfully
- [ ] Create a test role with permissions
- [ ] Configure object access settings via UI
- [ ] Share a record with another user
- [ ] Verify shared record appears in grantee's list
- [ ] Verify query scoping filters unauthorized records
- [ ] Test field-level permissions
- [ ] Test share expiration
- [ ] Test share revocation
- [ ] Test role-based access with conditions
## Performance Considerations
1. **Index Usage**: The migration creates proper indexes on foreign keys and commonly queried columns
2. **Query Scoping**: Uses SQL EXISTS subqueries for efficient filtering
3. **Ability Caching**: Consider caching abilities per request (already done via middleware)
4. **Batch Loading**: When checking multiple records, batch the share lookups
## Security Notes
⚠️ **Important**: Always use SQL query scoping for list endpoints. Never fetch all records and filter in application code.
**Best Practices**:
- Share creation requires ownership verification
- Only grantors can update/revoke shares
- Expired/revoked shares are excluded from queries
- Field-level permissions are enforced on write operations
## Documentation
Full documentation available in:
- [AUTHORIZATION_SYSTEM.md](./AUTHORIZATION_SYSTEM.md) - Comprehensive guide
- Inline code comments in all new files
- JSDoc comments on key functions
## Support
For questions or issues:
1. Check the documentation in `docs/AUTHORIZATION_SYSTEM.md`
2. Review example usage in the controllers
3. Examine the test cases (when added)

View File

@@ -1,173 +0,0 @@
# Softphone AI Assistant - Complete Implementation
## 🎉 Features Implemented
### ✅ Real-time AI Call Assistant
- **OpenAI Realtime API Integration** - Listens to live calls and provides suggestions
- **Audio Streaming** - Twilio Media Streams fork audio to backend for AI processing
- **Real-time Transcription** - Speech-to-text during calls
- **Smart Suggestions** - AI analyzes conversation and advises the agent
## 🔧 Architecture
### Backend Flow
```
Inbound Call → TwiML (<Start><Stream> + <Dial>)
→ Media Stream WebSocket → OpenAI Realtime API
→ AI Processing → Socket.IO → Frontend
```
### Key Components
1. **TwiML Structure** (`voice.controller.ts:226-234`)
- `<Start><Stream>` - Forks audio for AI processing
- `<Dial><Client>` - Connects call to agent's softphone
2. **OpenAI Integration** (`voice.service.ts:431-519`)
- WebSocket connection to `wss://api.openai.com/v1/realtime?model=gpt-4o-realtime-preview-2024-10-01`
- Session config with custom instructions for agent assistance
- Handles transcripts and generates suggestions
3. **AI Message Handler** (`voice.service.ts:609-707`)
- Processes OpenAI events (transcripts, suggestions, audio)
- Routes suggestions to frontend via Socket.IO
- Saves transcripts to database
4. **Voice Gateway** (`voice.gateway.ts:272-289`)
- `notifyAiTranscript()` - Real-time transcript chunks
- `notifyAiSuggestion()` - AI suggestions to agent
### Frontend Components
1. **Softphone Dialog** (`SoftphoneDialog.vue:104-135`)
- AI Assistant section with badge showing suggestion count
- Color-coded suggestions (blue=response, green=action, purple=insight)
- Animated highlight for newest suggestion
2. **Softphone Composable** (`useSoftphone.ts:515-535`)
- Socket.IO event handlers for `ai:suggestion` and `ai:transcript`
- Maintains history of last 10 suggestions
- Maintains history of last 50 transcript items
## 📋 AI Prompt Configuration
The AI is instructed to:
- **Listen, not talk** - It advises the agent, not the caller
- **Provide concise suggestions** - 1-2 sentences max
- **Use formatted output**:
- `💡 Suggestion: [advice]`
- `⚠️ Alert: [important notice]`
- `📋 Action: [CRM action]`
## 🎨 UI Features
### Suggestion Types
- **Response** (Blue) - Suggested replies or approaches
- **Action** (Green) - Recommended CRM actions
- **Insight** (Purple) - Important alerts or observations
### Visual Feedback
- Badge showing number of suggestions
- Newest suggestion pulses for attention
- Auto-scrolling suggestion list
- Timestamp on each suggestion
## 🔍 How to Monitor
### 1. Backend Logs
```bash
# Watch for AI events
docker logs -f neo-backend-1 | grep -E "AI|OpenAI|transcript|suggestion"
```
Key log markers:
- `📝 Transcript chunk:` - Real-time speech detection
- `✅ Final transcript:` - Complete transcript saved
- `💡 AI Suggestion:` - AI-generated advice
### 2. Database
```sql
-- View call transcripts
SELECT call_sid, ai_transcript, created_at
FROM calls
ORDER BY created_at DESC
LIMIT 5;
```
### 3. Frontend Console
- Open browser DevTools Console
- Watch for: "AI suggestion:", "AI transcript:"
## 🚀 Testing
1. **Make a test call** to your Twilio number
2. **Accept the call** in the softphone dialog
3. **Talk during the call** - Say something like "I need to schedule a follow-up"
4. **Watch the UI** - AI suggestions appear in real-time
5. **Check logs** - See transcription and suggestion generation
## 📊 Current Status
**Working**:
- Inbound calls ring softphone
- Media stream forks audio to backend
- OpenAI processes audio (1300+ packets/call)
- AI generates suggestions
- Suggestions appear in frontend
- Transcripts saved to database
## 🔧 Configuration
### Required Environment Variables
```env
# OpenAI API Key (set in tenant integrations config)
OPENAI_API_KEY=sk-...
# Optional overrides
OPENAI_MODEL=gpt-4o-realtime-preview-2024-10-01
OPENAI_VOICE=alloy
```
### Tenant Configuration
Set in Settings > Integrations:
- OpenAI API Key
- Model (optional)
- Voice (optional)
## 🎯 Next Steps (Optional Enhancements)
1. **CRM Tool Execution** - Implement actual tool calls (search contacts, create tasks)
2. **Audio Response** - Send OpenAI audio back to caller (two-way AI interaction)
3. **Sentiment Analysis** - Track call sentiment in real-time
4. **Call Summary** - Generate post-call summary automatically
5. **Custom Prompts** - Allow agents to customize AI instructions per call type
## 🐛 Troubleshooting
### No suggestions appearing?
1. Check OpenAI API key is configured
2. Verify WebSocket connection logs show "OpenAI Realtime connected"
3. Check frontend Socket.IO connection is established
4. Verify user ID matches between backend and frontend
### Transcripts not saving?
1. Check tenant database connection
2. Verify `calls` table has `ai_transcript` column
3. Check logs for "Failed to update transcript" errors
### OpenAI connection fails?
1. Verify API key is valid
2. Check model name is correct
3. Review WebSocket close codes in logs
## 📝 Files Modified
**Backend:**
- `/backend/src/voice/voice.service.ts` - OpenAI integration & AI message handling
- `/backend/src/voice/voice.controller.ts` - TwiML generation with stream fork
- `/backend/src/voice/voice.gateway.ts` - Socket.IO event emission
- `/backend/src/main.ts` - Media stream WebSocket handler
**Frontend:**
- `/frontend/components/SoftphoneDialog.vue` - AI suggestions UI
- `/frontend/composables/useSoftphone.ts` - Socket.IO event handlers

View File

@@ -0,0 +1,101 @@
/**
* Migration: Add authorization system (CASL + polymorphic sharing)
*
* This migration adds:
* 1. Access control fields to object_definitions
* 2. Field-level permissions to field_definitions
* 3. role_rules table for CASL rules storage
* 4. record_shares table for polymorphic per-record sharing
*/
exports.up = async function(knex) {
// 1. Add access control fields to object_definitions
await knex.schema.table('object_definitions', (table) => {
table.enum('access_model', ['public', 'owner', 'mixed']).defaultTo('owner');
table.boolean('public_read').defaultTo(false);
table.boolean('public_create').defaultTo(false);
table.boolean('public_update').defaultTo(false);
table.boolean('public_delete').defaultTo(false);
table.string('owner_field', 100).defaultTo('ownerId');
});
// 2. Add field-level permission columns to field_definitions
await knex.schema.table('field_definitions', (table) => {
table.boolean('default_readable').defaultTo(true);
table.boolean('default_writable').defaultTo(true);
});
// 3. Create role_rules table for storing CASL rules per role
await knex.schema.createTable('role_rules', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.uuid('role_id').notNullable();
table.json('rules_json').notNullable(); // Array of CASL rules
table.timestamps(true, true);
// Foreign keys
table.foreign('role_id')
.references('id')
.inTable('roles')
.onDelete('CASCADE');
// Indexes
table.index('role_id');
});
// 4. Create record_shares table for polymorphic per-record sharing
await knex.schema.createTable('record_shares', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.uuid('object_definition_id').notNullable();
table.string('record_id', 255).notNullable(); // String to support UUID/int uniformly
table.uuid('grantee_user_id').notNullable();
table.uuid('granted_by_user_id').notNullable();
table.json('actions').notNullable(); // Array like ["read"], ["read","update"]
table.json('fields').nullable(); // Optional field scoping
table.timestamp('expires_at').nullable();
table.timestamp('revoked_at').nullable();
table.timestamp('created_at').defaultTo(knex.fn.now());
// Foreign keys
table.foreign('object_definition_id')
.references('id')
.inTable('object_definitions')
.onDelete('CASCADE');
table.foreign('grantee_user_id')
.references('id')
.inTable('users')
.onDelete('CASCADE');
table.foreign('granted_by_user_id')
.references('id')
.inTable('users')
.onDelete('CASCADE');
// Indexes for efficient querying
table.index(['grantee_user_id', 'object_definition_id']);
table.index(['object_definition_id', 'record_id']);
table.unique(['object_definition_id', 'record_id', 'grantee_user_id']);
});
};
exports.down = async function(knex) {
// Drop tables in reverse order
await knex.schema.dropTableIfExists('record_shares');
await knex.schema.dropTableIfExists('role_rules');
// Remove columns from field_definitions
await knex.schema.table('field_definitions', (table) => {
table.dropColumn('default_readable');
table.dropColumn('default_writable');
});
// Remove columns from object_definitions
await knex.schema.table('object_definitions', (table) => {
table.dropColumn('access_model');
table.dropColumn('public_read');
table.dropColumn('public_create');
table.dropColumn('public_update');
table.dropColumn('public_delete');
table.dropColumn('owner_field');
});
};

View File

@@ -1,103 +0,0 @@
exports.up = function (knex) {
return knex.schema
// Add orgWideDefault to object_definitions
.alterTable('object_definitions', (table) => {
table
.enum('orgWideDefault', ['private', 'public_read', 'public_read_write'])
.defaultTo('private')
.notNullable();
})
// Create role_object_permissions table
.createTable('role_object_permissions', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.uuid('roleId').notNullable();
table.uuid('objectDefinitionId').notNullable();
table.boolean('canCreate').defaultTo(false);
table.boolean('canRead').defaultTo(false);
table.boolean('canEdit').defaultTo(false);
table.boolean('canDelete').defaultTo(false);
table.boolean('canViewAll').defaultTo(false);
table.boolean('canModifyAll').defaultTo(false);
table.timestamps(true, true);
table
.foreign('roleId')
.references('id')
.inTable('roles')
.onDelete('CASCADE');
table
.foreign('objectDefinitionId')
.references('id')
.inTable('object_definitions')
.onDelete('CASCADE');
table.unique(['roleId', 'objectDefinitionId']);
table.index(['roleId']);
table.index(['objectDefinitionId']);
})
// Create role_field_permissions table
.createTable('role_field_permissions', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.uuid('roleId').notNullable();
table.uuid('fieldDefinitionId').notNullable();
table.boolean('canRead').defaultTo(true);
table.boolean('canEdit').defaultTo(true);
table.timestamps(true, true);
table
.foreign('roleId')
.references('id')
.inTable('roles')
.onDelete('CASCADE');
table
.foreign('fieldDefinitionId')
.references('id')
.inTable('field_definitions')
.onDelete('CASCADE');
table.unique(['roleId', 'fieldDefinitionId']);
table.index(['roleId']);
table.index(['fieldDefinitionId']);
})
// Create record_shares table for sharing specific records
.createTable('record_shares', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.uuid('objectDefinitionId').notNullable();
table.uuid('recordId').notNullable();
table.uuid('granteeUserId').notNullable();
table.uuid('grantedByUserId').notNullable();
table.json('accessLevel').notNullable(); // { canRead, canEdit, canDelete }
table.timestamp('expiresAt').nullable();
table.timestamp('revokedAt').nullable();
table.timestamp('createdAt').defaultTo(knex.fn.now());
table.timestamp('updatedAt').defaultTo(knex.fn.now());
table
.foreign('objectDefinitionId')
.references('id')
.inTable('object_definitions')
.onDelete('CASCADE');
table
.foreign('granteeUserId')
.references('id')
.inTable('users')
.onDelete('CASCADE');
table
.foreign('grantedByUserId')
.references('id')
.inTable('users')
.onDelete('CASCADE');
table.index(['objectDefinitionId', 'recordId']);
table.index(['granteeUserId']);
table.index(['expiresAt']);
table.index(['revokedAt']);
});
};
exports.down = function (knex) {
return knex.schema
.dropTableIfExists('record_shares')
.dropTableIfExists('role_field_permissions')
.dropTableIfExists('role_object_permissions')
.alterTable('object_definitions', (table) => {
table.dropColumn('orgWideDefault');
});
};

View File

@@ -1,95 +0,0 @@
/**
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.up = async function(knex) {
// Check if layout_type column already exists (in case of partial migration)
const hasLayoutType = await knex.schema.hasColumn('page_layouts', 'layout_type');
// Check if the old index exists
const [indexes] = await knex.raw(`SHOW INDEX FROM page_layouts WHERE Key_name = 'page_layouts_object_id_is_default_index'`);
const hasOldIndex = indexes.length > 0;
// Check if foreign key exists
const [fks] = await knex.raw(`
SELECT CONSTRAINT_NAME FROM information_schema.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'page_layouts'
AND CONSTRAINT_TYPE = 'FOREIGN KEY'
AND CONSTRAINT_NAME = 'page_layouts_object_id_foreign'
`);
const hasForeignKey = fks.length > 0;
if (hasOldIndex) {
// First, drop the foreign key constraint that depends on the index (if it exists)
if (hasForeignKey) {
await knex.schema.alterTable('page_layouts', (table) => {
table.dropForeign(['object_id']);
});
}
// Now we can safely drop the old index
await knex.schema.alterTable('page_layouts', (table) => {
table.dropIndex(['object_id', 'is_default']);
});
}
// Add layout_type column if it doesn't exist
if (!hasLayoutType) {
await knex.schema.alterTable('page_layouts', (table) => {
// Add layout_type column to distinguish between detail/edit layouts and list view layouts
// Default to 'detail' for existing layouts
table.enum('layout_type', ['detail', 'list']).notNullable().defaultTo('detail').after('name');
});
}
// Check if new index exists
const [newIndexes] = await knex.raw(`SHOW INDEX FROM page_layouts WHERE Key_name = 'page_layouts_object_id_layout_type_is_default_index'`);
const hasNewIndex = newIndexes.length > 0;
if (!hasNewIndex) {
// Create new index including layout_type
await knex.schema.alterTable('page_layouts', (table) => {
table.index(['object_id', 'layout_type', 'is_default']);
});
}
// Re-check if foreign key exists (may have been dropped above or in previous attempt)
const [fksAfter] = await knex.raw(`
SELECT CONSTRAINT_NAME FROM information_schema.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'page_layouts'
AND CONSTRAINT_TYPE = 'FOREIGN KEY'
AND CONSTRAINT_NAME = 'page_layouts_object_id_foreign'
`);
if (fksAfter.length === 0) {
// Re-add the foreign key constraint
await knex.schema.alterTable('page_layouts', (table) => {
table.foreign('object_id').references('id').inTable('object_definitions').onDelete('CASCADE');
});
}
};
/**
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.down = async function(knex) {
// Drop the foreign key first
await knex.schema.alterTable('page_layouts', (table) => {
table.dropForeign(['object_id']);
});
// Drop the new index and column, restore old index
await knex.schema.alterTable('page_layouts', (table) => {
table.dropIndex(['object_id', 'layout_type', 'is_default']);
table.dropColumn('layout_type');
table.index(['object_id', 'is_default']);
});
// Re-add the foreign key constraint
await knex.schema.alterTable('page_layouts', (table) => {
table.foreign('object_id').references('id').inTable('object_definitions').onDelete('CASCADE');
});
};

View File

@@ -1,55 +0,0 @@
/**
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.up = async function (knex) {
// Create calls table for tracking voice calls
await knex.schema.createTable('calls', (table) => {
table.string('id', 36).primary();
table.string('call_sid', 100).unique().notNullable().comment('Twilio call SID');
table.enum('direction', ['inbound', 'outbound']).notNullable();
table.string('from_number', 20).notNullable();
table.string('to_number', 20).notNullable();
table.enum('status', [
'queued',
'ringing',
'in-progress',
'completed',
'busy',
'failed',
'no-answer',
'canceled'
]).notNullable().defaultTo('queued');
table.integer('duration_seconds').unsigned().nullable();
table.string('recording_url', 500).nullable();
table.text('ai_transcript').nullable().comment('Full transcript from OpenAI');
table.text('ai_summary').nullable().comment('AI-generated summary');
table.json('ai_insights').nullable().comment('Structured insights from AI');
table.string('user_id', 36).notNullable().comment('User who handled the call');
table.timestamp('started_at').nullable();
table.timestamp('ended_at').nullable();
table.timestamp('created_at').defaultTo(knex.fn.now());
table.timestamp('updated_at').defaultTo(knex.fn.now());
// Indexes
table.index('call_sid');
table.index('user_id');
table.index('status');
table.index('direction');
table.index(['created_at', 'user_id']);
// Foreign key to users table
table.foreign('user_id').references('id').inTable('users').onDelete('CASCADE');
});
console.log('✅ Created calls table');
};
/**
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.down = async function (knex) {
await knex.schema.dropTableIfExists('calls');
console.log('✅ Dropped calls table');
};

View File

@@ -1,207 +0,0 @@
exports.up = async function (knex) {
await knex.schema.createTable('contacts', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.string('firstName', 100).notNullable();
table.string('lastName', 100).notNullable();
table.uuid('accountId').notNullable();
table.timestamps(true, true);
table
.foreign('accountId')
.references('id')
.inTable('accounts')
.onDelete('CASCADE');
table.index(['accountId']);
table.index(['lastName', 'firstName']);
});
await knex.schema.createTable('contact_details', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.string('relatedObjectType', 100).notNullable();
table.uuid('relatedObjectId').notNullable();
table.string('detailType', 50).notNullable();
table.string('label', 100);
table.text('value').notNullable();
table.boolean('isPrimary').defaultTo(false);
table.timestamps(true, true);
table.index(['relatedObjectType', 'relatedObjectId']);
table.index(['detailType']);
});
const [contactObjectId] = await knex('object_definitions').insert({
id: knex.raw('(UUID())'),
apiName: 'Contact',
label: 'Contact',
pluralLabel: 'Contacts',
description: 'Standard Contact object',
isSystem: true,
isCustom: false,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
});
const contactObjectDefId =
contactObjectId ||
(await knex('object_definitions').where('apiName', 'Contact').first()).id;
await knex('field_definitions').insert([
{
id: knex.raw('(UUID())'),
objectDefinitionId: contactObjectDefId,
apiName: 'firstName',
label: 'First Name',
type: 'String',
length: 100,
isRequired: true,
isSystem: true,
isCustom: false,
displayOrder: 1,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
},
{
id: knex.raw('(UUID())'),
objectDefinitionId: contactObjectDefId,
apiName: 'lastName',
label: 'Last Name',
type: 'String',
length: 100,
isRequired: true,
isSystem: true,
isCustom: false,
displayOrder: 2,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
},
{
id: knex.raw('(UUID())'),
objectDefinitionId: contactObjectDefId,
apiName: 'accountId',
label: 'Account',
type: 'Reference',
referenceObject: 'Account',
isRequired: true,
isSystem: true,
isCustom: false,
displayOrder: 3,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
},
]);
const [contactDetailObjectId] = await knex('object_definitions').insert({
id: knex.raw('(UUID())'),
apiName: 'ContactDetail',
label: 'Contact Detail',
pluralLabel: 'Contact Details',
description: 'Polymorphic contact detail object',
isSystem: true,
isCustom: false,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
});
const contactDetailObjectDefId =
contactDetailObjectId ||
(await knex('object_definitions').where('apiName', 'ContactDetail').first())
.id;
const contactDetailRelationObjects = ['Account', 'Contact']
await knex('field_definitions').insert([
{
id: knex.raw('(UUID())'),
objectDefinitionId: contactDetailObjectDefId,
apiName: 'relatedObjectType',
label: 'Related Object Type',
type: 'PICKLIST',
length: 100,
isRequired: true,
isSystem: false,
isCustom: false,
displayOrder: 1,
ui_metadata: JSON.stringify({
options: contactDetailRelationObjects.map((value) => ({ label: value, value })),
}),
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
},
{
id: knex.raw('(UUID())'),
objectDefinitionId: contactDetailObjectDefId,
apiName: 'relatedObjectId',
label: 'Related Object ID',
type: 'LOOKUP',
length: 36,
isRequired: true,
isSystem: false,
isCustom: false,
displayOrder: 2,
ui_metadata: JSON.stringify({
relationObjects: contactDetailRelationObjects,
relationTypeField: 'relatedObjectType',
relationDisplayField: 'name',
}),
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
},
{
id: knex.raw('(UUID())'),
objectDefinitionId: contactDetailObjectDefId,
apiName: 'detailType',
label: 'Detail Type',
type: 'String',
length: 50,
isRequired: true,
isSystem: false,
isCustom: false,
displayOrder: 3,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
},
{
id: knex.raw('(UUID())'),
objectDefinitionId: contactDetailObjectDefId,
apiName: 'label',
label: 'Label',
type: 'String',
length: 100,
isSystem: false,
isCustom: false,
displayOrder: 4,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
},
{
id: knex.raw('(UUID())'),
objectDefinitionId: contactDetailObjectDefId,
apiName: 'value',
label: 'Value',
type: 'Text',
isRequired: true,
isSystem: false,
isCustom: false,
displayOrder: 5,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
},
{
id: knex.raw('(UUID())'),
objectDefinitionId: contactDetailObjectDefId,
apiName: 'isPrimary',
label: 'Primary',
type: 'Boolean',
isSystem: false,
isCustom: false,
displayOrder: 6,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
},
]);
};
exports.down = async function (knex) {
await knex.schema.dropTableIfExists('contact_details');
await knex.schema.dropTableIfExists('contacts');
};

View File

@@ -1,101 +0,0 @@
exports.up = async function (knex) {
const contactDetailObject = await knex('object_definitions')
.where({ apiName: 'ContactDetail' })
.first();
if (!contactDetailObject) return;
const relationObjects = ['Account', 'Contact'];
await knex('field_definitions')
.where({
objectDefinitionId: contactDetailObject.id,
apiName: 'relatedObjectType',
})
.update({
type: 'PICKLIST',
length: 100,
isSystem: false,
ui_metadata: JSON.stringify({
options: relationObjects.map((value) => ({ label: value, value })),
}),
updated_at: knex.fn.now(),
});
await knex('field_definitions')
.where({
objectDefinitionId: contactDetailObject.id,
apiName: 'relatedObjectId',
})
.update({
type: 'LOOKUP',
length: 36,
isSystem: false,
ui_metadata: JSON.stringify({
relationObjects,
relationTypeField: 'relatedObjectType',
relationDisplayField: 'name',
}),
updated_at: knex.fn.now(),
});
await knex('field_definitions')
.whereIn('apiName', [
'detailType',
'label',
'value',
'isPrimary',
])
.andWhere({ objectDefinitionId: contactDetailObject.id })
.update({
isSystem: false,
updated_at: knex.fn.now(),
});
};
exports.down = async function (knex) {
const contactDetailObject = await knex('object_definitions')
.where({ apiName: 'ContactDetail' })
.first();
if (!contactDetailObject) return;
await knex('field_definitions')
.where({
objectDefinitionId: contactDetailObject.id,
apiName: 'relatedObjectType',
})
.update({
type: 'String',
length: 100,
isSystem: true,
ui_metadata: null,
updated_at: knex.fn.now(),
});
await knex('field_definitions')
.where({
objectDefinitionId: contactDetailObject.id,
apiName: 'relatedObjectId',
})
.update({
type: 'String',
length: 36,
isSystem: true,
ui_metadata: null,
updated_at: knex.fn.now(),
});
await knex('field_definitions')
.whereIn('apiName', [
'detailType',
'label',
'value',
'isPrimary',
])
.andWhere({ objectDefinitionId: contactDetailObject.id })
.update({
isSystem: true,
updated_at: knex.fn.now(),
});
};

View File

@@ -1,45 +0,0 @@
exports.up = async function (knex) {
const contactDetailObject = await knex('object_definitions')
.where({ apiName: 'ContactDetail' })
.first();
if (!contactDetailObject) return;
await knex('field_definitions')
.where({ objectDefinitionId: contactDetailObject.id })
.whereIn('apiName', [
'relatedObjectType',
'relatedObjectId',
'detailType',
'label',
'value',
'isPrimary',
])
.update({
isSystem: false,
updated_at: knex.fn.now(),
});
};
exports.down = async function (knex) {
const contactDetailObject = await knex('object_definitions')
.where({ apiName: 'ContactDetail' })
.first();
if (!contactDetailObject) return;
await knex('field_definitions')
.where({ objectDefinitionId: contactDetailObject.id })
.whereIn('apiName', [
'relatedObjectType',
'relatedObjectId',
'detailType',
'label',
'value',
'isPrimary',
])
.update({
isSystem: true,
updated_at: knex.fn.now(),
});
};

View File

@@ -1,62 +0,0 @@
exports.up = async function (knex) {
// Add ownerId column to contacts
await knex.schema.alterTable('contacts', (table) => {
table.uuid('ownerId');
table
.foreign('ownerId')
.references('id')
.inTable('users')
.onDelete('SET NULL');
table.index(['ownerId']);
});
// Add ownerId field definition metadata for Contact object
const contactObject = await knex('object_definitions')
.where('apiName', 'Contact')
.first();
if (contactObject) {
const existingField = await knex('field_definitions')
.where({
objectDefinitionId: contactObject.id,
apiName: 'ownerId',
})
.first();
if (!existingField) {
await knex('field_definitions').insert({
id: knex.raw('(UUID())'),
objectDefinitionId: contactObject.id,
apiName: 'ownerId',
label: 'Owner',
type: 'Reference',
referenceObject: 'User',
isSystem: true,
isCustom: false,
displayOrder: 4,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
});
}
}
};
exports.down = async function (knex) {
const contactObject = await knex('object_definitions')
.where('apiName', 'Contact')
.first();
if (contactObject) {
await knex('field_definitions')
.where({
objectDefinitionId: contactObject.id,
apiName: 'ownerId',
})
.delete();
}
await knex.schema.alterTable('contacts', (table) => {
table.dropForeign(['ownerId']);
table.dropColumn('ownerId');
});
};

View File

@@ -1,55 +0,0 @@
/**
* Creates the saved_list_views table.
* Each row stores a named, reusable search/filter configuration for a specific
* CRM object type. Views can be private to the owning user or shared with the
* whole tenant.
*
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.up = function (knex) {
return knex.schema.createTable('saved_list_views', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
// Human-readable name given by the user (or AI-suggested)
table.string('name').notNullable();
// The object this view belongs to (e.g. "Dog", "Contact")
table.string('object_api_name').notNullable();
// The user who created/owns this view
table.uuid('user_id').notNullable();
// When true the view is visible to all users in the tenant
table.boolean('is_shared').notNullable().defaultTo(false);
// Strategy is always "query" for saved views (keyword views are not saved)
table.string('strategy').notNullable().defaultTo('query');
// Resolved filters as JSON array of AiSearchFilter objects
table.json('filters').notNullable();
// Optional sort: { field: string, direction: "asc" | "desc" }
table.json('sort').nullable();
// AI-generated plain-language explanation of what this view shows
table.text('description').nullable();
table.timestamps(true, true);
// Foreign key to users
table.foreign('user_id').references('id').inTable('users').onDelete('CASCADE');
// Primary lookup: all views for an object visible to a user
table.index(['object_api_name', 'user_id']);
table.index(['object_api_name', 'is_shared']);
});
};
/**
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.down = function (knex) {
return knex.schema.dropTableIfExists('saved_list_views');
};

View File

@@ -1,35 +0,0 @@
/**
* Inserts a system object_definition row for SavedListView.
* This allows saved_list_views records to be shared via record_shares
* (which requires a valid objectDefinitionId FK).
*
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.up = async function (knex) {
// Only insert if it doesn't already exist (idempotent)
const existing = await knex('object_definitions')
.where({ apiName: 'SavedListView' })
.first();
if (!existing) {
await knex('object_definitions').insert({
apiName: 'SavedListView',
label: 'Saved List View',
pluralLabel: 'Saved List Views',
description: 'System object for sharing saved list views via record_shares',
isSystem: true,
isCustom: false,
});
}
};
/**
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.down = async function (knex) {
await knex('object_definitions')
.where({ apiName: 'SavedListView' })
.delete();
};

View File

@@ -1,30 +0,0 @@
/**
* Add 'alias' and virtual 'name' column to users table.
*
* - alias: a user-editable display name / nickname
* - name: a generated column that returns COALESCE(alias, CONCAT(firstName, ' ', lastName), email)
* so that lookup fields referencing User.name always resolve.
*/
exports.up = function (knex) {
return knex.schema.alterTable('users', (table) => {
table.string('alias', 255).nullable().after('lastName');
table.string('name', 512).nullable().after('alias');
}).then(() => {
// Backfill existing rows: name = alias, or firstName + lastName, or email
return knex.raw(`
UPDATE users
SET name = COALESCE(
NULLIF(alias, ''),
NULLIF(TRIM(CONCAT(COALESCE(firstName, ''), ' ', COALESCE(lastName, ''))), ''),
email
)
`);
});
};
exports.down = function (knex) {
return knex.schema.alterTable('users', (table) => {
table.dropColumn('name');
table.dropColumn('alias');
});
};

View File

@@ -1,93 +0,0 @@
/**
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.up = async function (knex) {
await knex.schema.createTable('comments', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.string('parent_object_api_name').notNullable();
table.uuid('parent_record_id').notNullable();
table.uuid('author_user_id').notNullable();
table.text('content').notNullable();
table.timestamps(true, true);
table.foreign('author_user_id').references('id').inTable('users').onDelete('CASCADE');
table.index(['parent_object_api_name', 'parent_record_id'], 'comments_parent_idx');
table.index(['author_user_id'], 'comments_author_idx');
});
await knex.schema.createTable('semantic_documents', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.string('entity_type').notNullable();
table.uuid('entity_id').notNullable();
table.string('title').nullable();
table.text('narrative').nullable();
table.json('metadata').nullable();
table.json('source_summary').nullable();
table.timestamps(true, true);
table.unique(['entity_type', 'entity_id'], {
indexName: 'semantic_documents_entity_unique',
});
table.index(['entity_type'], 'semantic_documents_type_idx');
});
await knex.schema.createTable('semantic_chunks', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.uuid('semantic_document_id').notNullable();
table.integer('chunk_index').notNullable();
table.string('source_kind').notNullable().defaultTo('base_record');
table.uuid('source_ref_id').nullable();
table.text('text').notNullable();
table.json('metadata').nullable();
table.timestamps(true, true);
table.foreign('semantic_document_id').references('id').inTable('semantic_documents').onDelete('CASCADE');
table.unique(['semantic_document_id', 'chunk_index'], {
indexName: 'semantic_chunks_doc_index_unique',
});
table.index(['semantic_document_id'], 'semantic_chunks_document_idx');
table.index(['source_kind'], 'semantic_chunks_source_kind_idx');
});
await knex.schema.createTable('semantic_links', (table) => {
table.uuid('id').primary().defaultTo(knex.raw('(UUID())'));
table.string('source_entity_type', 100).notNullable();
table.uuid('source_entity_id').notNullable();
table.string('target_entity_type', 100).notNullable();
table.uuid('target_entity_id').notNullable();
table.string('link_type', 100).notNullable().defaultTo('related_to');
table.string('status').notNullable().defaultTo('suggested');
table.string('origin').notNullable().defaultTo('semantic');
table.decimal('confidence', 5, 4).notNullable().defaultTo(0);
table.text('reason').nullable();
table.json('evidence').nullable();
table.uuid('suggested_by_user_id').nullable();
table.uuid('reviewed_by_user_id').nullable();
table.timestamp('reviewed_at').nullable();
table.timestamps(true, true);
table.foreign('suggested_by_user_id').references('id').inTable('users').onDelete('SET NULL');
table.foreign('reviewed_by_user_id').references('id').inTable('users').onDelete('SET NULL');
table.unique(
['source_entity_type', 'source_entity_id', 'target_entity_type', 'target_entity_id', 'link_type'],
{ indexName: 'semantic_links_unique_pair_type' },
);
table.index(['source_entity_type', 'source_entity_id'], 'semantic_links_source_idx');
table.index(['target_entity_type', 'target_entity_id'], 'semantic_links_target_idx');
table.index(['status'], 'semantic_links_status_idx');
});
};
/**
* @param { import("knex").Knex } knex
* @returns { Promise<void> }
*/
exports.down = async function (knex) {
await knex.schema.dropTableIfExists('semantic_links');
await knex.schema.dropTableIfExists('semantic_chunks');
await knex.schema.dropTableIfExists('semantic_documents');
await knex.schema.dropTableIfExists('comments');
};

1307
backend/package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -27,10 +27,6 @@
},
"dependencies": {
"@casl/ability": "^6.7.5",
"@fastify/websocket": "^10.0.1",
"@langchain/core": "^1.1.15",
"@langchain/langgraph": "^1.0.15",
"@langchain/openai": "^1.2.1",
"@nestjs/bullmq": "^10.1.0",
"@nestjs/common": "^10.3.0",
"@nestjs/config": "^3.1.1",
@@ -38,28 +34,20 @@
"@nestjs/jwt": "^10.2.0",
"@nestjs/passport": "^10.0.3",
"@nestjs/platform-fastify": "^10.3.0",
"@nestjs/platform-socket.io": "^10.4.20",
"@nestjs/serve-static": "^4.0.2",
"@nestjs/websockets": "^10.4.20",
"@prisma/client": "^5.8.0",
"bcrypt": "^5.1.1",
"bullmq": "^5.1.0",
"class-transformer": "^0.5.1",
"class-validator": "^0.14.1",
"deepagents": "^1.5.0",
"ioredis": "^5.3.2",
"knex": "^3.1.0",
"langchain": "^1.2.10",
"mysql2": "^3.15.3",
"objection": "^3.1.5",
"openai": "^6.15.0",
"objection-authorize": "^5.0.2",
"passport": "^0.7.0",
"passport-jwt": "^4.0.1",
"reflect-metadata": "^0.2.1",
"rxjs": "^7.8.1",
"socket.io": "^4.8.3",
"twilio": "^5.11.1",
"ws": "^8.18.3"
"rxjs": "^7.8.1"
},
"devDependencies": {
"@nestjs/cli": "^10.3.0",

View File

@@ -1,2 +0,0 @@
-- AlterTable
ALTER TABLE `tenants` ADD COLUMN `integrationsConfig` JSON NULL;

View File

@@ -32,7 +32,6 @@ model Tenant {
dbName String // Database name
dbUsername String // Database username
dbPassword String // Encrypted database password
integrationsConfig Json? // Encrypted JSON config for external services (Twilio, OpenAI, etc.)
status String @default("active") // active, suspended, deleted
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt

View File

@@ -20,14 +20,14 @@ model User {
password String
firstName String?
lastName String?
alias String?
name String?
isActive Boolean @default(true)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
userRoles UserRole[]
accounts Account[]
sharesGranted RecordShare[] @relation("GrantedShares")
sharesReceived RecordShare[] @relation("ReceivedShares")
@@map("users")
}
@@ -43,6 +43,7 @@ model Role {
userRoles UserRole[]
rolePermissions RolePermission[]
roleRules RoleRule[]
@@unique([name, guardName])
@@map("roles")
@@ -92,6 +93,20 @@ model RolePermission {
@@map("role_permissions")
}
// CASL Rules for Roles
model RoleRule {
id String @id @default(uuid())
roleId String
rulesJson Json @map("rules_json")
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
role Role @relation(fields: [roleId], references: [id], onDelete: Cascade)
@@index([roleId])
@@map("role_rules")
}
// Object Definition (Metadata)
model ObjectDefinition {
id String @id @default(uuid())
@@ -101,11 +116,19 @@ model ObjectDefinition {
description String? @db.Text
isSystem Boolean @default(false)
isCustom Boolean @default(true)
// Authorization fields
accessModel String @default("owner") // 'public' | 'owner' | 'mixed'
publicRead Boolean @default(false)
publicCreate Boolean @default(false)
publicUpdate Boolean @default(false)
publicDelete Boolean @default(false)
ownerField String @default("ownerId")
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")
fields FieldDefinition[]
pages AppPage[]
recordShares RecordShare[]
@@map("object_definitions")
}
@@ -128,6 +151,9 @@ model FieldDefinition {
isCustom Boolean @default(true)
displayOrder Int @default(0)
uiMetadata Json? @map("ui_metadata")
// Field-level permissions
defaultReadable Boolean @default(true)
defaultWritable Boolean @default(true)
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")
@@ -138,6 +164,29 @@ model FieldDefinition {
@@map("field_definitions")
}
// Polymorphic per-record sharing
model RecordShare {
id String @id @default(uuid())
objectDefinitionId String
recordId String
granteeUserId String
grantedByUserId String
actions Json // Array like ["read"], ["read","update"]
fields Json? // Optional field scoping
expiresAt DateTime? @map("expires_at")
revokedAt DateTime? @map("revoked_at")
createdAt DateTime @default(now()) @map("created_at")
objectDefinition ObjectDefinition @relation(fields: [objectDefinitionId], references: [id], onDelete: Cascade)
granteeUser User @relation("ReceivedShares", fields: [granteeUserId], references: [id], onDelete: Cascade)
grantedByUser User @relation("GrantedShares", fields: [grantedByUserId], references: [id], onDelete: Cascade)
@@unique([objectDefinitionId, recordId, granteeUserId])
@@index([granteeUserId, objectDefinitionId])
@@index([objectDefinitionId, recordId])
@@map("record_shares")
}
// Example static object: Account
model Account {
id String @id @default(uuid())
@@ -148,41 +197,11 @@ model Account {
updatedAt DateTime @updatedAt
owner User @relation(fields: [ownerId], references: [id])
contacts Contact[]
@@index([ownerId])
@@map("accounts")
}
model Contact {
id String @id @default(uuid())
firstName String
lastName String
accountId String
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
account Account @relation(fields: [accountId], references: [id], onDelete: Cascade)
@@index([accountId])
@@map("contacts")
}
model ContactDetail {
id String @id @default(uuid())
relatedObjectType String
relatedObjectId String
detailType String
label String?
value String
isPrimary Boolean @default(false)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([relatedObjectType, relatedObjectId])
@@map("contact_details")
}
// Application Builder
model App {
id String @id @default(uuid())

View File

@@ -1,181 +0,0 @@
import { Knex } from 'knex';
import * as knexLib from 'knex';
/**
* Create a Knex connection for tenant database
*/
function createKnexConnection(database: string): Knex {
return knexLib.default({
client: 'mysql2',
connection: {
host: process.env.DB_HOST || 'db',
port: parseInt(process.env.DB_PORT || '3306'),
user: 'root',
password: 'asjdnfqTash37faggT',
database: database,
},
});
}
interface RoleWithPermissions {
name: string;
description: string;
objectPermissions: {
[objectApiName: string]: {
canCreate: boolean;
canRead: boolean;
canEdit: boolean;
canDelete: boolean;
canViewAll: boolean;
canModifyAll: boolean;
};
};
}
const DEFAULT_ROLES: RoleWithPermissions[] = [
{
name: 'System Administrator',
description: 'Full access to all objects and records. Can view and modify all data.',
objectPermissions: {
'*': {
canCreate: true,
canRead: true,
canEdit: true,
canDelete: true,
canViewAll: true,
canModifyAll: true,
},
},
},
{
name: 'Standard User',
description: 'Can create, read, edit, and delete own records. Respects OWD settings.',
objectPermissions: {
'*': {
canCreate: true,
canRead: true,
canEdit: true,
canDelete: true,
canViewAll: false,
canModifyAll: false,
},
},
},
{
name: 'Read Only',
description: 'Can only read records based on OWD settings. No create, edit, or delete.',
objectPermissions: {
'*': {
canCreate: false,
canRead: true,
canEdit: false,
canDelete: false,
canViewAll: false,
canModifyAll: false,
},
},
},
];
async function seedRolesForTenant(knex: Knex, tenantName: string) {
console.log(`\n🌱 Seeding roles for tenant: ${tenantName}`);
// Get all object definitions
const objectDefinitions = await knex('object_definitions').select('id', 'apiName');
for (const roleData of DEFAULT_ROLES) {
// Check if role already exists
const existingRole = await knex('roles')
.where({ name: roleData.name })
.first();
let roleId: string;
if (existingRole) {
console.log(` Role "${roleData.name}" already exists, skipping...`);
roleId = existingRole.id;
} else {
// Create role
await knex('roles').insert({
name: roleData.name,
guardName: 'api',
description: roleData.description,
});
// Get the inserted role
const newRole = await knex('roles')
.where({ name: roleData.name })
.first();
roleId = newRole.id;
console.log(` ✅ Created role: ${roleData.name}`);
}
// Create object permissions for all objects
const wildcardPermissions = roleData.objectPermissions['*'];
for (const objectDef of objectDefinitions) {
// Check if permission already exists
const existingPermission = await knex('role_object_permissions')
.where({
roleId: roleId,
objectDefinitionId: objectDef.id,
})
.first();
if (!existingPermission) {
await knex('role_object_permissions').insert({
roleId: roleId,
objectDefinitionId: objectDef.id,
canCreate: wildcardPermissions.canCreate,
canRead: wildcardPermissions.canRead,
canEdit: wildcardPermissions.canEdit,
canDelete: wildcardPermissions.canDelete,
canViewAll: wildcardPermissions.canViewAll,
canModifyAll: wildcardPermissions.canModifyAll,
});
}
}
console.log(` 📋 Set permissions for ${objectDefinitions.length} objects`);
}
}
async function seedAllTenants() {
console.log('🚀 Starting role seeding for all tenants...\n');
// For now, seed the main tenant database
const databases = ['tenant_tenant1'];
let successCount = 0;
let errorCount = 0;
for (const database of databases) {
try {
const knex = createKnexConnection(database);
await seedRolesForTenant(knex, database);
await knex.destroy();
successCount++;
} catch (error) {
console.error(`${database}: Seeding failed:`, error.message);
errorCount++;
}
}
console.log('\n============================================================');
console.log('📊 Seeding Summary');
console.log('============================================================');
console.log(`✅ Successful: ${successCount}`);
console.log(`❌ Failed: ${errorCount}`);
if (errorCount === 0) {
console.log('\n🎉 All tenant roles seeded successfully!');
}
}
seedAllTenants()
.then(() => process.exit(0))
.catch((error) => {
console.error('Unhandled error:', error);
process.exit(1);
});

View File

@@ -1,49 +0,0 @@
import { Body, Controller, Post, UseGuards } from '@nestjs/common';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { CurrentUser } from '../auth/current-user.decorator';
import { TenantId } from '../tenant/tenant.decorator';
import { AiAssistantService } from './ai-assistant.service';
import { AiChatRequestDto } from './dto/ai-chat.dto';
import { AiSearchRequestDto } from './dto/ai-search.dto';
@Controller('ai')
@UseGuards(JwtAuthGuard)
export class AiAssistantController {
constructor(private readonly aiAssistantService: AiAssistantService) {}
@Post('chat')
async chat(
@TenantId() tenantId: string,
@CurrentUser() user: any,
@Body() payload: AiChatRequestDto,
) {
return this.aiAssistantService.handleChat(
tenantId,
user.userId,
payload.message,
payload.history,
payload.context,
);
}
@Post('search')
async search(
@TenantId() tenantId: string,
@CurrentUser() user: any,
@Body() payload: AiSearchRequestDto,
) {
return this.aiAssistantService.searchRecords(
tenantId,
user.userId,
payload,
);
}
@Post('suggest-view-name')
async suggestViewName(
@TenantId() tenantId: string,
@Body() payload: { objectLabel: string; filters: any[]; explanation?: string },
) {
return this.aiAssistantService.suggestViewName(tenantId, payload);
}
}

View File

@@ -1,14 +0,0 @@
import { Module } from '@nestjs/common';
import { AiAssistantController } from './ai-assistant.controller';
import { AiAssistantService } from './ai-assistant.service';
import { ObjectModule } from '../object/object.module';
import { PageLayoutModule } from '../page-layout/page-layout.module';
import { TenantModule } from '../tenant/tenant.module';
import { MeilisearchModule } from '../search/meilisearch.module';
@Module({
imports: [ObjectModule, PageLayoutModule, TenantModule, MeilisearchModule],
controllers: [AiAssistantController],
providers: [AiAssistantService],
})
export class AiAssistantModule {}

File diff suppressed because it is too large Load Diff

View File

@@ -1,111 +0,0 @@
export interface AiChatMessage {
role: 'user' | 'assistant';
text: string;
}
export interface AiChatContext {
objectApiName?: string;
view?: string;
recordId?: string;
route?: string;
}
export interface AiAssistantReply {
reply: string;
action?: 'create_record' | 'collect_fields' | 'clarify' | 'plan_complete' | 'plan_pending';
missingFields?: string[];
record?: any;
records?: any[]; // Multiple records when plan execution completes
plan?: RecordCreationPlan;
}
// ============================================
// Entity Discovery Types
// ============================================
export interface EntityFieldInfo {
apiName: string;
label: string;
type: string;
isRequired: boolean;
isSystem: boolean;
referenceObject?: string; // For LOOKUP fields, the target entity
description?: string;
}
export interface EntityRelationship {
fieldApiName: string;
fieldLabel: string;
targetEntity: string;
relationshipType: 'lookup' | 'master-detail' | 'polymorphic';
}
export interface EntityInfo {
apiName: string;
label: string;
pluralLabel?: string;
description?: string;
fields: EntityFieldInfo[];
requiredFields: string[]; // Field apiNames that are required
relationships: EntityRelationship[];
}
export interface SystemEntities {
entities: EntityInfo[];
entityByApiName: Record<string, EntityInfo>; // Changed from Map for state serialization
loadedAt: number;
}
// ============================================
// Planning Types
// ============================================
export interface PlannedRecord {
id: string; // Temporary ID for planning (e.g., "temp_account_1")
entityApiName: string;
entityLabel: string;
fields: Record<string, any>;
resolvedFields?: Record<string, any>; // Fields after dependency resolution
missingRequiredFields: string[];
dependsOn: string[]; // IDs of other planned records this depends on
status: 'pending' | 'ready' | 'created' | 'failed';
createdRecordId?: string; // Actual ID after creation
wasExisting?: boolean; // True if record already existed in database
error?: string;
}
export interface RecordCreationPlan {
id: string;
records: PlannedRecord[];
executionOrder: string[]; // Ordered list of planned record IDs
status: 'building' | 'incomplete' | 'ready' | 'executing' | 'completed' | 'failed';
createdRecords: any[];
errors: string[];
}
// ============================================
// State Types
// ============================================
export interface AiAssistantState {
message: string;
messages?: any[]; // BaseMessage[] from langchain - used when invoked by Deep Agent
history?: AiChatMessage[];
context: AiChatContext;
// Entity discovery
systemEntities?: SystemEntities;
// Planning
plan?: RecordCreationPlan;
// Legacy fields (kept for compatibility during transition)
objectDefinition?: any;
pageLayout?: any;
extractedFields?: Record<string, any>;
requiredFields?: string[];
missingFields?: string[];
action?: AiAssistantReply['action'];
record?: any;
reply?: string;
}

View File

@@ -1,36 +0,0 @@
import { Type } from 'class-transformer';
import { IsNotEmpty, IsObject, IsOptional, IsString, ValidateNested } from 'class-validator';
import { AiChatMessageDto } from './ai-chat.message.dto';
export class AiChatContextDto {
@IsOptional()
@IsString()
objectApiName?: string;
@IsOptional()
@IsString()
view?: string;
@IsOptional()
@IsString()
recordId?: string;
@IsOptional()
@IsString()
route?: string;
}
export class AiChatRequestDto {
@IsString()
@IsNotEmpty()
message: string;
@IsOptional()
@IsObject()
context?: AiChatContextDto;
@IsOptional()
@ValidateNested({ each: true })
@Type(() => AiChatMessageDto)
history?: AiChatMessageDto[];
}

View File

@@ -1,10 +0,0 @@
import { IsIn, IsNotEmpty, IsString } from 'class-validator';
export class AiChatMessageDto {
@IsIn(['user', 'assistant'])
role: 'user' | 'assistant';
@IsString()
@IsNotEmpty()
text: string;
}

View File

@@ -1,22 +0,0 @@
import { Type } from 'class-transformer';
import { IsNotEmpty, IsOptional, IsString, IsNumber } from 'class-validator';
export class AiSearchRequestDto {
@IsString()
@IsNotEmpty()
objectApiName: string;
@IsString()
@IsNotEmpty()
query: string;
@IsOptional()
@Type(() => Number)
@IsNumber()
page?: number;
@IsOptional()
@Type(() => Number)
@IsNumber()
pageSize?: number;
}

View File

@@ -10,14 +10,14 @@ export class AppBuilderService {
// Runtime endpoints
async getApps(tenantId: string, userId: string) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
// For now, return all apps
// In production, you'd filter by user permissions
return App.query(knex).withGraphFetched('pages').orderBy('label', 'asc');
}
async getApp(tenantId: string, slug: string, userId: string) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
const app = await App.query(knex)
.findOne({ slug })
.withGraphFetched('pages');
@@ -35,7 +35,7 @@ export class AppBuilderService {
pageSlug: string,
userId: string,
) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
const app = await this.getApp(tenantId, appSlug, userId);
const page = await AppPage.query(knex).findOne({
@@ -52,12 +52,12 @@ export class AppBuilderService {
// Setup endpoints
async getAllApps(tenantId: string) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
return App.query(knex).withGraphFetched('pages').orderBy('label', 'asc');
}
async getAppForSetup(tenantId: string, slug: string) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
const app = await App.query(knex)
.findOne({ slug })
.withGraphFetched('pages');
@@ -77,7 +77,7 @@ export class AppBuilderService {
description?: string;
},
) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
return App.query(knex).insert({
...data,
displayOrder: 0,
@@ -92,7 +92,7 @@ export class AppBuilderService {
description?: string;
},
) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
const app = await this.getAppForSetup(tenantId, slug);
return App.query(knex).patchAndFetchById(app.id, data);
@@ -109,7 +109,7 @@ export class AppBuilderService {
sortOrder?: number;
},
) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
const app = await this.getAppForSetup(tenantId, appSlug);
return AppPage.query(knex).insert({
@@ -133,7 +133,7 @@ export class AppBuilderService {
sortOrder?: number;
},
) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
const app = await this.getAppForSetup(tenantId, appSlug);
const page = await AppPage.query(knex).findOne({

View File

@@ -1,6 +1,5 @@
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { BullModule } from '@nestjs/bullmq';
import { PrismaModule } from './prisma/prisma.module';
import { TenantModule } from './tenant/tenant.module';
import { AuthModule } from './auth/auth.module';
@@ -8,22 +7,12 @@ import { RbacModule } from './rbac/rbac.module';
import { ObjectModule } from './object/object.module';
import { AppBuilderModule } from './app-builder/app-builder.module';
import { PageLayoutModule } from './page-layout/page-layout.module';
import { VoiceModule } from './voice/voice.module';
import { AiAssistantModule } from './ai-assistant/ai-assistant.module';
import { SavedListViewModule } from './saved-list-view/saved-list-view.module';
import { KnowledgeModule } from './knowledge/knowledge.module';
@Module({
imports: [
ConfigModule.forRoot({
isGlobal: true,
}),
BullModule.forRoot({
connection: {
host: process.env.REDIS_HOST || 'platform-redis',
port: parseInt(process.env.REDIS_PORT || '6379', 10),
},
}),
PrismaModule,
TenantModule,
AuthModule,
@@ -31,10 +20,6 @@ import { KnowledgeModule } from './knowledge/knowledge.module';
ObjectModule,
AppBuilderModule,
PageLayoutModule,
VoiceModule,
AiAssistantModule,
SavedListViewModule,
KnowledgeModule,
],
})
export class AppModule {}

View File

@@ -0,0 +1,207 @@
import { Injectable } from '@nestjs/common';
import { Ability, AbilityBuilder, AbilityClass, ExtractSubjectType, InferSubjects, createMongoAbility } from '@casl/ability';
import { User } from '../models/user.model';
import { ObjectDefinition } from '../models/object-definition.model';
import { FieldDefinition } from '../models/field-definition.model';
import { RoleRule } from '../models/role-rule.model';
import { RecordShare } from '../models/record-share.model';
import { UserRole } from '../models/user-role.model';
import { Knex } from 'knex';
// Define actions
export type Action = 'read' | 'create' | 'update' | 'delete' | 'share';
// Define subjects - can be string (object type key) or model class
export type Subjects = InferSubjects<any> | 'all';
export type AppAbility = Ability<[Action, Subjects]>;
@Injectable()
export class AbilityFactory {
/**
* Build CASL Ability for a user
* Rules come from 3 layers:
* 1. Global object rules (from object_definitions + object_fields)
* 2. Role rules (from role_rules)
* 3. Share rules (from record_shares for this user)
*/
async buildForUser(user: User, knex: Knex): Promise<AppAbility> {
const { can, cannot, build } = new AbilityBuilder<AppAbility>(
createMongoAbility as any,
);
// 1. Load global object rules
await this.addGlobalRules(user, knex, can, cannot);
// 2. Load role rules
await this.addRoleRules(user, knex, can);
// 3. Load share rules
await this.addShareRules(user, knex, can);
return build({
// Optional: detect subject type from instance
detectSubjectType: (item) => {
if (typeof item === 'string') return item;
return item.constructor?.name || 'unknown';
},
});
}
/**
* Add global rules from object_definitions and object_fields
*/
private async addGlobalRules(
user: User,
knex: Knex,
can: any,
cannot: any,
) {
const objectDefs = await knex<ObjectDefinition>('object_definitions').select('*');
for (const objDef of objectDefs) {
const subject = objDef.apiName;
// Handle public access
if (objDef.publicRead) {
can('read', subject);
}
if (objDef.publicCreate) {
can('create', subject);
}
if (objDef.publicUpdate) {
can('update', subject);
}
if (objDef.publicDelete) {
can('delete', subject);
}
// Handle owner-based access
if (objDef.accessModel === 'owner' || objDef.accessModel === 'mixed') {
const ownerCondition = { [objDef.ownerField]: user.id };
can('read', subject, ownerCondition);
can('update', subject, ownerCondition);
can('delete', subject, ownerCondition);
can('share', subject, ownerCondition); // Owner can share their records
}
// Load field-level permissions for this object
const fields = await knex<FieldDefinition>('field_definitions')
.where('objectDefinitionId', objDef.id)
.select('*');
// Build field lists
const readableFields = fields
.filter((f) => f.defaultReadable)
.map((f) => f.apiName);
const writableFields = fields
.filter((f) => f.defaultWritable)
.map((f) => f.apiName);
// Add field-level rules if we have field restrictions
if (fields.length > 0) {
// For read, limit to readable fields
if (readableFields.length > 0) {
can('read', subject, readableFields);
}
// For update/create, limit to writable fields
if (writableFields.length > 0) {
can(['update', 'create'], subject, writableFields);
}
}
}
}
/**
* Add role-based rules from role_rules
*/
private async addRoleRules(user: User, knex: Knex, can: any) {
// Get user's roles
const userRoles = await knex<UserRole>('user_roles')
.where('userId', user.id)
.select('roleId');
if (userRoles.length === 0) return;
const roleIds = userRoles.map((ur) => ur.roleId);
// Get all role rules for these roles
const roleRules = await knex<RoleRule>('role_rules')
.whereIn('roleId', roleIds)
.select('*');
for (const roleRule of roleRules) {
// Parse and add each rule from the JSON
const rules = roleRule.rulesJson;
if (Array.isArray(rules)) {
rules.forEach((rule) => {
if (rule.inverted) {
// Handle "cannot" rules
// CASL format: { action, subject, conditions?, fields?, inverted: true }
// We'd need to properly parse this - for now, skip inverted rules in factory
} else {
// Handle "can" rules
const { action, subject, conditions, fields } = rule;
if (fields && fields.length > 0) {
can(action, subject, fields, conditions);
} else if (conditions) {
can(action, subject, conditions);
} else {
can(action, subject);
}
}
});
}
}
}
/**
* Add per-record sharing rules from record_shares
*/
private async addShareRules(user: User, knex: Knex, can: any) {
const now = new Date();
// Get all active shares for this user (grantee)
const shares = await knex<RecordShare>('record_shares')
.where('granteeUserId', user.id)
.whereNull('revokedAt')
.where(function () {
this.whereNull('expiresAt').orWhere('expiresAt', '>', now);
})
.select('*');
// Also need to join with object_definitions to get the apiName (subject)
const sharesWithObjects = await knex('record_shares')
.join('object_definitions', 'record_shares.objectDefinitionId', 'object_definitions.id')
.where('record_shares.granteeUserId', user.id)
.whereNull('record_shares.revokedAt')
.where(function () {
this.whereNull('record_shares.expiresAt').orWhere('record_shares.expiresAt', '>', now);
})
.select(
'record_shares.*',
'object_definitions.apiName as objectApiName',
);
for (const share of sharesWithObjects) {
const subject = share.objectApiName;
const actions = Array.isArray(share.actions) ? share.actions : JSON.parse(share.actions);
const fields = share.fields ? (Array.isArray(share.fields) ? share.fields : JSON.parse(share.fields)) : null;
// Create condition: record must match the shared recordId
const condition = { id: share.recordId };
for (const action of actions) {
if (fields && fields.length > 0) {
// Field-scoped share
can(action, subject, fields, condition);
} else {
// Full record share
can(action, subject, condition);
}
}
}
}
}

View File

@@ -1,19 +1,15 @@
import {
Controller,
Post,
Get,
Body,
UnauthorizedException,
HttpCode,
HttpStatus,
Req,
UseGuards,
} from '@nestjs/common';
import { IsEmail, IsString, MinLength, IsOptional } from 'class-validator';
import { AuthService } from './auth.service';
import { TenantId } from '../tenant/tenant.decorator';
import { JwtAuthGuard } from './jwt-auth.guard';
import { CurrentUser } from './current-user.decorator';
class LoginDto {
@IsEmail()
@@ -115,15 +111,4 @@ export class AuthController {
// This endpoint exists for consistency and potential future enhancements
return { message: 'Logged out successfully' };
}
@UseGuards(JwtAuthGuard)
@Get('me')
async me(@CurrentUser() user: any, @TenantId() tenantId: string) {
// Return the current authenticated user info
return {
id: user.userId,
email: user.email,
tenantId: tenantId || user.tenantId,
};
}
}

View File

@@ -6,6 +6,8 @@ import { AuthService } from './auth.service';
import { AuthController } from './auth.controller';
import { JwtStrategy } from './jwt.strategy';
import { TenantModule } from '../tenant/tenant.module';
import { AbilityFactory } from './ability.factory';
import { AbilitiesGuard } from './guards/abilities.guard';
@Module({
imports: [
@@ -19,8 +21,8 @@ import { TenantModule } from '../tenant/tenant.module';
}),
}),
],
providers: [AuthService, JwtStrategy],
providers: [AuthService, JwtStrategy, AbilityFactory, AbilitiesGuard],
controllers: [AuthController],
exports: [AuthService],
exports: [AuthService, AbilityFactory, AbilitiesGuard],
})
export class AuthModule {}

View File

@@ -29,7 +29,7 @@ export class AuthService {
}
// Otherwise, validate as tenant user
const tenantDb = await this.tenantDbService.getTenantKnexById(tenantId);
const tenantDb = await this.tenantDbService.getTenantKnex(tenantId);
const user = await tenantDb('users')
.where({ email })
@@ -113,7 +113,7 @@ export class AuthService {
}
// Otherwise, register as tenant user
const tenantDb = await this.tenantDbService.getTenantKnexById(tenantId);
const tenantDb = await this.tenantDbService.getTenantKnex(tenantId);
const hashedPassword = await bcrypt.hash(password, 10);

View File

@@ -0,0 +1,24 @@
import { createParamDecorator, ExecutionContext } from '@nestjs/common';
import { AppAbility } from '../ability.factory';
/**
* Decorator to inject the current user's ability into a route handler
* Usage: @CurrentAbility() ability: AppAbility
*/
export const CurrentAbility = createParamDecorator(
(data: unknown, ctx: ExecutionContext): AppAbility => {
const request = ctx.switchToHttp().getRequest();
return request.ability;
},
);
/**
* Decorator to inject the current user into a route handler
* Usage: @CurrentUser() user: User
*/
export const CurrentUser = createParamDecorator(
(data: unknown, ctx: ExecutionContext) => {
const request = ctx.switchToHttp().getRequest();
return request.user;
},
);

View File

@@ -0,0 +1,10 @@
import { SetMetadata } from '@nestjs/common';
import { Action } from '../ability.factory';
import { CHECK_ABILITY_KEY, RequiredRule } from '../guards/abilities.guard';
/**
* Decorator to check abilities
* Usage: @CheckAbility({ action: 'read', subject: 'Post' })
*/
export const CheckAbility = (...rules: RequiredRule[]) =>
SetMetadata(CHECK_ABILITY_KEY, rules);

View File

@@ -0,0 +1,51 @@
import { Injectable, CanActivate, ExecutionContext, ForbiddenException } from '@nestjs/common';
import { Reflector } from '@nestjs/core';
import { Action, AppAbility } from '../ability.factory';
export interface RequiredRule {
action: Action;
subject: string;
}
/**
* Key for metadata
*/
export const CHECK_ABILITY_KEY = 'check_ability';
/**
* Guard that checks CASL abilities
* Use with @CheckAbility() decorator
*/
@Injectable()
export class AbilitiesGuard implements CanActivate {
constructor(private reflector: Reflector) {}
async canActivate(context: ExecutionContext): Promise<boolean> {
const rules = this.reflector.get<RequiredRule[]>(
CHECK_ABILITY_KEY,
context.getHandler(),
) || [];
if (rules.length === 0) {
return true; // No rules specified, allow
}
const request = context.switchToHttp().getRequest();
const ability: AppAbility = request.ability;
if (!ability) {
throw new ForbiddenException('Ability not found on request');
}
// Check all rules
for (const rule of rules) {
if (!ability.can(rule.action, rule.subject)) {
throw new ForbiddenException(
`You don't have permission to ${rule.action} ${rule.subject}`,
);
}
}
return true;
}
}

View File

@@ -0,0 +1,24 @@
import { Injectable, NestMiddleware, Inject } from '@nestjs/common';
import { Request, Response, NextFunction } from 'express';
import { AbilityFactory } from '../ability.factory';
import { Knex } from 'knex';
/**
* Middleware to build and attach CASL ability to request
* Must run after authentication middleware
*/
@Injectable()
export class AbilityMiddleware implements NestMiddleware {
constructor(
private readonly abilityFactory: AbilityFactory,
@Inject('KnexConnection') private readonly knex: Knex,
) {}
async use(req: Request & { user?: any; ability?: any }, res: Response, next: NextFunction) {
if (req.user) {
// Build ability for authenticated user
req.ability = await this.abilityFactory.buildForUser(req.user, this.knex);
}
next();
}
}

View File

@@ -0,0 +1,145 @@
import { QueryBuilder, Model } from 'objection';
import { User } from '../models/user.model';
import { ObjectDefinition } from '../models/object-definition.model';
import { Knex } from 'knex';
/**
* Query scoping utilities for authorization
* Apply SQL-level filtering to ensure users only see records they have access to
*/
export interface AuthScopeOptions {
user: User;
objectDefinition: ObjectDefinition;
action: 'read' | 'update' | 'delete';
knex: Knex;
}
/**
* Apply authorization scope to a query builder
* This implements the SQL equivalent of the CASL ability checks
*
* Rules:
* 1. If object is public_{action} => allow all
* 2. If object is owner/mixed => allow owned OR shared
*/
export function applyAuthScope<M extends Model>(
query: QueryBuilder<M, M[]>,
options: AuthScopeOptions,
): QueryBuilder<M, M[]> {
const { user, objectDefinition, action, knex } = options;
// If public access for this action, no restrictions
if (
(action === 'read' && objectDefinition.publicRead) ||
(action === 'update' && objectDefinition.publicUpdate) ||
(action === 'delete' && objectDefinition.publicDelete)
) {
return query;
}
// Otherwise, apply owner + share logic
const ownerField = objectDefinition.ownerField || 'ownerId';
const tableName = query.modelClass().tableName;
return query.where((builder) => {
// Owner condition
builder.where(`${tableName}.${ownerField}`, user.id);
// OR shared condition
builder.orWhereExists((subquery) => {
subquery
.from('record_shares')
.join('object_definitions', 'record_shares.object_definition_id', 'object_definitions.id')
.whereRaw('record_shares.record_id = ??', [`${tableName}.id`])
.where('record_shares.grantee_user_id', user.id)
.where('object_definitions.id', objectDefinition.id)
.whereNull('record_shares.revoked_at')
.where(function () {
this.whereNull('record_shares.expires_at')
.orWhere('record_shares.expires_at', '>', knex.fn.now());
})
.whereRaw("JSON_CONTAINS(record_shares.actions, ?)", [JSON.stringify(action)]);
});
});
}
/**
* Apply read scope - most common use case
*/
export function applyReadScope<M extends Model>(
query: QueryBuilder<M, M[]>,
user: User,
objectDefinition: ObjectDefinition,
knex: Knex,
): QueryBuilder<M, M[]> {
return applyAuthScope(query, { user, objectDefinition, action: 'read', knex });
}
/**
* Apply update scope
*/
export function applyUpdateScope<M extends Model>(
query: QueryBuilder<M, M[]>,
user: User,
objectDefinition: ObjectDefinition,
knex: Knex,
): QueryBuilder<M, M[]> {
return applyAuthScope(query, { user, objectDefinition, action: 'update', knex });
}
/**
* Apply delete scope
*/
export function applyDeleteScope<M extends Model>(
query: QueryBuilder<M, M[]>,
user: User,
objectDefinition: ObjectDefinition,
knex: Knex,
): QueryBuilder<M, M[]> {
return applyAuthScope(query, { user, objectDefinition, action: 'delete', knex });
}
/**
* Check if user can access a specific record
* This is for single-record operations
*/
export async function canAccessRecord(
recordId: string,
user: User,
objectDefinition: ObjectDefinition,
action: 'read' | 'update' | 'delete',
knex: Knex,
): Promise<boolean> {
// If public access for this action
if (
(action === 'read' && objectDefinition.publicRead) ||
(action === 'update' && objectDefinition.publicUpdate) ||
(action === 'delete' && objectDefinition.publicDelete)
) {
return true;
}
const ownerField = objectDefinition.ownerField || 'ownerId';
// Check if user owns the record (we need the table name, which we can't easily get here)
// This function is meant to be used with a fetched record
// For now, we'll check shares only
// Check if there's a valid share
const now = new Date();
const share = await knex('record_shares')
.where({
objectDefinitionId: objectDefinition.id,
recordId: recordId,
granteeUserId: user.id,
})
.whereNull('revokedAt')
.where(function () {
this.whereNull('expiresAt').orWhere('expiresAt', '>', now);
})
.whereRaw("JSON_CONTAINS(actions, ?)", [JSON.stringify(action)])
.first();
return !!share;
}

View File

@@ -1,89 +0,0 @@
export type SemanticProjectionInput = {
objectApiName: string;
record: Record<string, any>;
objectDefinition?: any;
comments: Array<{ id: string; content: string; author_user_id: string; created_at?: string }>;
};
export type SemanticProjection = {
entityType: string;
entityId: string;
title: string;
narrative: string;
/** Plain text used for embedding — no 'key: value' labels, no comments (chunker handles those separately). */
embeddingNarrative: string;
metadata: Record<string, any>;
sourceSummary: {
includedFieldCount: number;
includedCommentCount: number;
includesComments: boolean;
};
};
export interface SemanticProjectionAdapter {
supports(objectApiName: string): boolean;
buildProjection(input: SemanticProjectionInput): SemanticProjection;
}
const EXCLUDED_FIELDS = new Set([
'id',
'created_at',
'updated_at',
'ownerId',
'owner_id',
'tenantId',
'tenant_id',
]);
export class DefaultSemanticProjectionAdapter implements SemanticProjectionAdapter {
supports(): boolean {
return true;
}
buildProjection(input: SemanticProjectionInput): SemanticProjection {
const fieldEntries = Object.entries(input.record || {}).filter(([key, value]) => {
if (EXCLUDED_FIELDS.has(key)) return false;
if (value === null || value === undefined || value === '') return false;
return ['string', 'number', 'boolean'].includes(typeof value);
});
const title =
input.record?.name ||
input.record?.title ||
input.record?.subject ||
`${input.objectApiName} ${input.record?.id || ''}`.trim();
const fieldNarrative = fieldEntries
.map(([key, value]) => `${key}: ${String(value)}`)
.join('\n');
const commentNarrative = (input.comments || [])
.map((comment, index) => `Comment ${index + 1}: ${comment.content}`)
.join('\n');
const narrative = [fieldNarrative, commentNarrative].filter(Boolean).join('\n\n');
// Plain values only — no 'key:' prefixes. Comments are handled separately by the chunker.
const embeddingNarrative = fieldEntries
.map(([, value]) => String(value))
.join('\n');
return {
entityType: input.objectApiName,
entityId: input.record.id,
title,
narrative,
embeddingNarrative,
metadata: {
objectApiName: input.objectApiName,
hasComments: (input.comments || []).length > 0,
},
sourceSummary: {
includedFieldCount: fieldEntries.length,
includedCommentCount: (input.comments || []).length,
includesComments: (input.comments || []).length > 0,
},
};
}
}

View File

@@ -1,24 +0,0 @@
import { IsNotEmpty, IsOptional, IsString, MaxLength, MinLength } from 'class-validator';
export class CreateCommentDto {
@IsString()
@IsNotEmpty()
parentObjectApiName: string;
@IsString()
@IsNotEmpty()
parentRecordId: string;
@IsString()
@MinLength(1)
@MaxLength(10000)
content: string;
}
export class UpdateCommentDto {
@IsOptional()
@IsString()
@MinLength(1)
@MaxLength(10000)
content?: string;
}

View File

@@ -1,52 +0,0 @@
import { IsIn, IsNumber, IsObject, IsOptional, IsString, Max, Min } from 'class-validator';
export const SEMANTIC_LINK_STATUSES = ['suggested', 'approved', 'rejected', 'dismissed'] as const;
export const SEMANTIC_LINK_ORIGINS = ['manual', 'semantic', 'llm', 'hybrid', 'rule_based'] as const;
export class ReviewSemanticLinkDto {
@IsString()
@IsIn(SEMANTIC_LINK_STATUSES)
status: (typeof SEMANTIC_LINK_STATUSES)[number];
}
export class UpsertSemanticLinkDto {
@IsString()
sourceEntityType: string;
@IsString()
sourceEntityId: string;
@IsString()
targetEntityType: string;
@IsString()
targetEntityId: string;
@IsOptional()
@IsString()
linkType?: string;
@IsOptional()
@IsString()
@IsIn(SEMANTIC_LINK_STATUSES)
status?: (typeof SEMANTIC_LINK_STATUSES)[number];
@IsOptional()
@IsString()
@IsIn(SEMANTIC_LINK_ORIGINS)
origin?: (typeof SEMANTIC_LINK_ORIGINS)[number];
@IsOptional()
@IsNumber()
@Min(0)
@Max(1)
confidence?: number;
@IsOptional()
@IsString()
reason?: string;
@IsOptional()
@IsObject()
evidence?: Record<string, any>;
}

View File

@@ -1,124 +0,0 @@
import {
Body,
Controller,
Delete,
Get,
Param,
Patch,
Post,
Query,
UseGuards,
} from '@nestjs/common';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { CurrentUser } from '../auth/current-user.decorator';
import { TenantId } from '../tenant/tenant.decorator';
import { CreateCommentDto, UpdateCommentDto } from './dto/comment.dto';
import { ReviewSemanticLinkDto } from './dto/semantic-link.dto';
import { CommentService } from './services/comment.service';
import { SemanticOrchestratorService } from './services/semantic-orchestrator.service';
import { SemanticLinkService } from './services/semantic-link.service';
import { TenantDatabaseService } from '../tenant/tenant-database.service';
@Controller('knowledge')
@UseGuards(JwtAuthGuard)
export class KnowledgeController {
constructor(
private readonly commentService: CommentService,
private readonly semanticOrchestratorService: SemanticOrchestratorService,
private readonly semanticLinkService: SemanticLinkService,
private readonly tenantDbService: TenantDatabaseService,
) {}
@Get('comments/:objectApiName/:recordId')
async getComments(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Param('recordId') recordId: string,
) {
return this.commentService.listComments(tenantId, objectApiName, recordId);
}
@Post('comments')
async createComment(
@TenantId() tenantId: string,
@Body() dto: CreateCommentDto,
@CurrentUser() user: any,
) {
return this.commentService.createComment(tenantId, dto, user.userId);
}
@Patch('comments/:id')
async updateComment(
@TenantId() tenantId: string,
@Param('id') id: string,
@Body() dto: UpdateCommentDto,
@CurrentUser() user: any,
) {
return this.commentService.updateComment(tenantId, id, dto, user.userId);
}
@Delete('comments/:id')
async deleteComment(
@TenantId() tenantId: string,
@Param('id') id: string,
@CurrentUser() user: any,
) {
return this.commentService.deleteComment(tenantId, id, user.userId);
}
@Post('semantic/refresh/:objectApiName/:recordId')
async refreshSemantic(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Param('recordId') recordId: string,
@CurrentUser() user: any,
) {
return this.semanticOrchestratorService.refreshRecord(
tenantId,
objectApiName,
recordId,
user.userId,
'manual_refresh',
);
}
@Post('semantic/reindex/:objectApiName')
async reindexObject(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@CurrentUser() user: any,
@Query('limit') limit?: string,
) {
const parsedLimit = Number.isFinite(Number(limit)) ? Number(limit) : 250;
return this.semanticOrchestratorService.reindexObject(
tenantId,
objectApiName,
user.userId,
parsedLimit,
);
}
@Get('semantic/links/:objectApiName/:recordId')
async listLinks(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Param('recordId') recordId: string,
@Query('status') status?: string,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
return this.semanticLinkService.listForRecord(knex, objectApiName, recordId, status);
}
@Patch('semantic/links/:id/review')
async reviewLink(
@TenantId() tenantId: string,
@Param('id') id: string,
@Body() dto: ReviewSemanticLinkDto,
@CurrentUser() user: any,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
return this.semanticLinkService.reviewLink(knex, id, dto.status, user.userId);
}
}

View File

@@ -1,31 +0,0 @@
import { Module } from '@nestjs/common';
import { BullModule } from '@nestjs/bullmq';
import { KnowledgeController } from './knowledge.controller';
import { CommentService } from './services/comment.service';
import { SemanticOrchestratorService } from './services/semantic-orchestrator.service';
import { SemanticChunkerService } from './services/semantic-chunker.service';
import { SemanticLinkService } from './services/semantic-link.service';
import { SemanticRefreshQueueService } from './services/semantic-refresh-queue.service';
import { SemanticRefreshProcessor } from './semantic-refresh.processor';
import { TenantModule } from '../tenant/tenant.module';
import { MeilisearchModule } from '../search/meilisearch.module';
import { SEMANTIC_REFRESH_QUEUE } from './semantic-refresh.constants';
@Module({
imports: [
TenantModule,
MeilisearchModule,
BullModule.registerQueue({ name: SEMANTIC_REFRESH_QUEUE }),
],
controllers: [KnowledgeController],
providers: [
CommentService,
SemanticOrchestratorService,
SemanticChunkerService,
SemanticLinkService,
SemanticRefreshQueueService,
SemanticRefreshProcessor,
],
exports: [SemanticOrchestratorService, SemanticRefreshQueueService],
})
export class KnowledgeModule {}

View File

@@ -1,3 +0,0 @@
export const SEMANTIC_REFRESH_QUEUE = 'semantic-refresh';
export const SEMANTIC_REFRESH_JOB = 'refresh-record';

View File

@@ -1,45 +0,0 @@
import { Processor, WorkerHost } from '@nestjs/bullmq';
import { Logger } from '@nestjs/common';
import { Job } from 'bullmq';
import { SemanticOrchestratorService } from './services/semantic-orchestrator.service';
import { SEMANTIC_REFRESH_QUEUE } from './semantic-refresh.constants';
export type SemanticRefreshJobData = {
tenantId: string;
objectApiName: string;
recordId: string;
userId?: string;
trigger: string;
};
@Processor(SEMANTIC_REFRESH_QUEUE)
export class SemanticRefreshProcessor extends WorkerHost {
private readonly logger = new Logger(SemanticRefreshProcessor.name);
constructor(
private readonly semanticOrchestratorService: SemanticOrchestratorService,
) {
super();
}
async process(job: Job<SemanticRefreshJobData>): Promise<void> {
const { tenantId, objectApiName, recordId, userId, trigger } = job.data;
this.logger.log(
`Processing semantic refresh: ${objectApiName}:${recordId} trigger=${trigger}`,
);
try {
await this.semanticOrchestratorService.refreshRecord(
tenantId,
objectApiName,
recordId,
userId,
trigger,
);
} catch (error) {
this.logger.error(
`Semantic refresh failed: ${objectApiName}:${recordId} trigger=${trigger} error=${error.message}`,
);
throw error; // Let BullMQ handle retries
}
}
}

View File

@@ -1,115 +0,0 @@
import { ForbiddenException, Injectable, NotFoundException } from '@nestjs/common';
import { TenantDatabaseService } from '../../tenant/tenant-database.service';
import { CreateCommentDto, UpdateCommentDto } from '../dto/comment.dto';
import { SemanticRefreshQueueService } from './semantic-refresh-queue.service';
@Injectable()
export class CommentService {
constructor(
private readonly tenantDbService: TenantDatabaseService,
private readonly semanticRefreshQueue: SemanticRefreshQueueService,
) {}
async listComments(tenantId: string, parentObjectApiName: string, parentRecordId: string) {
const knex = await this.getKnex(tenantId);
return knex('comments')
.where({
parent_object_api_name: parentObjectApiName,
parent_record_id: parentRecordId,
})
.orderBy('created_at', 'desc');
}
async createComment(tenantId: string, dto: CreateCommentDto, userId: string) {
const knex = await this.getKnex(tenantId);
const [created] = await knex('comments')
.insert({
parent_object_api_name: dto.parentObjectApiName,
parent_record_id: dto.parentRecordId,
author_user_id: userId,
content: dto.content,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
})
.returning('*');
console.log(
`[Knowledge] Comment created: ${dto.parentObjectApiName}:${dto.parentRecordId} by ${userId}`,
);
await this.semanticRefreshQueue.enqueue(
tenantId,
dto.parentObjectApiName,
dto.parentRecordId,
userId,
'comment_created',
);
return created;
}
async updateComment(tenantId: string, commentId: string, dto: UpdateCommentDto, userId: string) {
const knex = await this.getKnex(tenantId);
const existing = await knex('comments').where({ id: commentId }).first();
if (!existing) {
throw new NotFoundException('Comment not found');
}
if (existing.author_user_id !== userId) {
throw new ForbiddenException('Only the author can edit this comment');
}
await knex('comments')
.where({ id: commentId })
.update({
...(dto.content ? { content: dto.content } : {}),
updated_at: knex.fn.now(),
});
console.log(
`[Knowledge] Comment updated: ${existing.parent_object_api_name}:${existing.parent_record_id} by ${userId}`,
);
await this.semanticRefreshQueue.enqueue(
tenantId,
existing.parent_object_api_name,
existing.parent_record_id,
userId,
'comment_updated',
);
return knex('comments').where({ id: commentId }).first();
}
async deleteComment(tenantId: string, commentId: string, userId: string) {
const knex = await this.getKnex(tenantId);
const existing = await knex('comments').where({ id: commentId }).first();
if (!existing) {
throw new NotFoundException('Comment not found');
}
if (existing.author_user_id !== userId) {
throw new ForbiddenException('Only the author can delete this comment');
}
await knex('comments').where({ id: commentId }).delete();
console.log(
`[Knowledge] Comment deleted: ${existing.parent_object_api_name}:${existing.parent_record_id} by ${userId}`,
);
await this.semanticRefreshQueue.enqueue(
tenantId,
existing.parent_object_api_name,
existing.parent_record_id,
userId,
'comment_deleted',
);
return { success: true };
}
private async getKnex(tenantId: string) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
return this.tenantDbService.getTenantKnexById(resolvedTenantId);
}
}

View File

@@ -1,20 +0,0 @@
import { SemanticChunkerService } from './semantic-chunker.service';
describe('SemanticChunkerService', () => {
let service: SemanticChunkerService;
beforeEach(() => {
service = new SemanticChunkerService();
});
it('creates chunks from base narrative and comments', () => {
const chunks = service.chunkText('Intro paragraph\n\nSecond paragraph', [
{ id: 'c-1', content: 'Comment body' },
]);
expect(chunks).toHaveLength(3);
expect(chunks[0].sourceKind).toBe('base_record');
expect(chunks[2].sourceKind).toBe('comment');
expect(chunks[2].sourceRefId).toBe('c-1');
});
});

View File

@@ -1,71 +0,0 @@
import { Injectable } from '@nestjs/common';
export type SemanticChunk = {
chunkIndex: number;
sourceKind: 'base_record' | 'comment' | 'mixed';
sourceRefId: string | null;
text: string;
metadata: Record<string, any>;
};
@Injectable()
export class SemanticChunkerService {
chunkText(
baseNarrative: string,
comments: Array<{ id: string; content: string }>,
): SemanticChunk[] {
const chunks: SemanticChunk[] = [];
const baseParts = this.splitText(baseNarrative);
for (const [index, text] of baseParts.entries()) {
chunks.push({
chunkIndex: chunks.length,
sourceKind: 'base_record',
sourceRefId: null,
text,
metadata: { section: 'base', localIndex: index },
});
}
for (const comment of comments || []) {
const commentParts = this.splitText(comment.content);
for (const [index, text] of commentParts.entries()) {
chunks.push({
chunkIndex: chunks.length,
sourceKind: 'comment',
sourceRefId: comment.id,
text,
metadata: { section: 'comment', localIndex: index, commentId: comment.id },
});
}
}
return chunks;
}
private splitText(text: string): string[] {
const normalized = (text || '').trim();
if (!normalized) return [];
const paragraphs = normalized
.split(/\n{2,}/)
.map((part) => part.trim())
.filter(Boolean);
const chunks: string[] = [];
for (const paragraph of paragraphs) {
if (paragraph.length <= 500) {
chunks.push(paragraph);
continue;
}
let cursor = 0;
while (cursor < paragraph.length) {
chunks.push(paragraph.slice(cursor, cursor + 500).trim());
cursor += 500;
}
}
return chunks.filter(Boolean);
}
}

View File

@@ -1,20 +0,0 @@
import { SemanticLinkService } from './semantic-link.service';
describe('SemanticLinkService', () => {
let service: SemanticLinkService;
beforeEach(() => {
service = new SemanticLinkService();
});
it('normalizes undirected pairs in deterministic order', () => {
const normalized = service.normalizeUndirectedPair('Contact', 'b-id', 'Account', 'a-id');
expect(normalized).toEqual({
sourceEntityType: 'Account',
sourceEntityId: 'a-id',
targetEntityType: 'Contact',
targetEntityId: 'b-id',
});
});
});

View File

@@ -1,186 +0,0 @@
import { Injectable, NotFoundException } from '@nestjs/common';
export type SemanticLinkUpsertInput = {
sourceEntityType: string;
sourceEntityId: string;
targetEntityType: string;
targetEntityId: string;
linkType?: string;
status?: string;
origin?: string;
confidence?: number;
reason?: string;
evidence?: Record<string, any>;
suggestedByUserId?: string | null;
};
@Injectable()
export class SemanticLinkService {
normalizeUndirectedPair(
sourceEntityType: string,
sourceEntityId: string,
targetEntityType: string,
targetEntityId: string,
) {
const sourceKey = `${sourceEntityType}:${sourceEntityId}`;
const targetKey = `${targetEntityType}:${targetEntityId}`;
if (sourceKey <= targetKey) {
return {
sourceEntityType,
sourceEntityId,
targetEntityType,
targetEntityId,
};
}
return {
sourceEntityType: targetEntityType,
sourceEntityId: targetEntityId,
targetEntityType: sourceEntityType,
targetEntityId: sourceEntityId,
};
}
async upsertSuggestedLink(knex: any, input: SemanticLinkUpsertInput) {
const normalized = this.normalizeUndirectedPair(
input.sourceEntityType,
input.sourceEntityId,
input.targetEntityType,
input.targetEntityId,
);
const payload = {
source_entity_type: normalized.sourceEntityType,
source_entity_id: normalized.sourceEntityId,
target_entity_type: normalized.targetEntityType,
target_entity_id: normalized.targetEntityId,
link_type: input.linkType || 'related_to',
status: input.status || 'suggested',
origin: input.origin || 'semantic',
confidence: input.confidence ?? 0,
reason: input.reason || null,
evidence: input.evidence ? JSON.stringify(input.evidence) : null,
suggested_by_user_id: input.suggestedByUserId || null,
updated_at: knex.fn.now(),
created_at: knex.fn.now(),
};
await knex('semantic_links')
.insert(payload)
.onConflict([
'source_entity_type',
'source_entity_id',
'target_entity_type',
'target_entity_id',
'link_type',
])
.merge({
status: knex.raw("IF(status = 'approved', status, VALUES(status))"),
origin: payload.origin,
confidence: knex.raw('GREATEST(confidence, VALUES(confidence))'),
reason: payload.reason,
evidence: payload.evidence,
updated_at: knex.fn.now(),
});
}
async listForRecord(knex: any, entityType: string, entityId: string, status?: string) {
const query = knex('semantic_links')
.where((builder: any) => {
builder
.where({ source_entity_type: entityType, source_entity_id: entityId })
.orWhere({ target_entity_type: entityType, target_entity_id: entityId });
})
.orderBy('updated_at', 'desc');
if (status) {
query.andWhere({ status });
}
const links = await query;
if (!links.length) return links;
const typeSet = new Set<string>();
for (const link of links) {
typeSet.add(link.source_entity_type);
typeSet.add(link.target_entity_type);
}
const definitions = await knex('object_definitions')
.whereIn('apiName', Array.from(typeSet))
.select('apiName', 'label', 'pluralLabel', 'tableName', 'fields');
const definitionByType = new Map<string, any>(
definitions.map((item: any) => [item.apiName, item]),
);
const displayNameCache = new Map<string, string>();
const getDisplayField = (definition: any) => {
let fields = [];
if (Array.isArray(definition?.fields)) {
fields = definition.fields;
} else if (typeof definition?.fields === 'string') {
try {
fields = JSON.parse(definition.fields);
} catch {
fields = [];
}
}
if (fields.some((field: any) => field?.apiName === 'name')) return 'name';
const textField = fields.find((field: any) =>
['STRING', 'TEXT', 'EMAIL'].includes(String(field?.type || '').toUpperCase()),
);
return textField?.apiName || 'id';
};
const resolveTableName = (definition: any) => {
if (definition?.tableName) return definition.tableName;
if (definition?.pluralLabel) {
return String(definition.pluralLabel).toLowerCase().replace(/[^a-z0-9]+/g, '_');
}
return `${String(definition?.apiName || '').toLowerCase()}s`;
};
const loadDisplayName = async (type: string, id: string) => {
const cacheKey = `${type}:${id}`;
if (displayNameCache.has(cacheKey)) return displayNameCache.get(cacheKey);
const definition = definitionByType.get(type);
if (!definition) {
displayNameCache.set(cacheKey, id);
return id;
}
const tableName = resolveTableName(definition);
const displayField = getDisplayField(definition);
const record = await knex(tableName).where({ id }).first();
const display = record?.[displayField] ? String(record[displayField]) : id;
displayNameCache.set(cacheKey, display);
return display;
};
for (const link of links) {
link.source_entity_label = definitionByType.get(link.source_entity_type)?.label || link.source_entity_type;
link.target_entity_label = definitionByType.get(link.target_entity_type)?.label || link.target_entity_type;
link.source_entity_name = await loadDisplayName(link.source_entity_type, link.source_entity_id);
link.target_entity_name = await loadDisplayName(link.target_entity_type, link.target_entity_id);
}
return links;
}
async reviewLink(knex: any, linkId: string, status: string, reviewerUserId: string) {
const updated = await knex('semantic_links')
.where({ id: linkId })
.update({
status,
reviewed_by_user_id: reviewerUserId,
reviewed_at: knex.fn.now(),
updated_at: knex.fn.now(),
});
if (!updated) {
throw new NotFoundException('Semantic link not found');
}
return knex('semantic_links').where({ id: linkId }).first();
}
}

View File

@@ -1,540 +0,0 @@
import { Injectable, Logger } from '@nestjs/common';
import { HumanMessage, SystemMessage } from '@langchain/core/messages';
import { ChatOpenAI } from '@langchain/openai';
import { TenantDatabaseService } from '../../tenant/tenant-database.service';
import { MeilisearchService } from '../../search/meilisearch.service';
import { getCentralPrisma } from '../../prisma/central-prisma.service';
import { OpenAIConfig } from '../../voice/interfaces/integration-config.interface';
import { randomUUID } from 'crypto';
import {
DefaultSemanticProjectionAdapter,
SemanticProjectionAdapter,
} from '../adapters/semantic-projection.adapter';
import { SemanticChunkerService } from './semantic-chunker.service';
import { SemanticLinkService } from './semantic-link.service';
@Injectable()
export class SemanticOrchestratorService {
private readonly logger = new Logger(SemanticOrchestratorService.name);
private readonly adapters: SemanticProjectionAdapter[] = [new DefaultSemanticProjectionAdapter()];
private readonly defaultEmbeddingModel =
process.env.OPENAI_EMBEDDING_MODEL || 'text-embedding-3-small';
private readonly semanticEmbedderName = 'default';
private readonly MIN_CONFIDENCE_BASE = 0.7;
private readonly MIN_CONFIDENCE_COMMENT = 0.52;
private readonly defaultChatModel = process.env.OPENAI_CHAT_MODEL || 'gpt-4o-mini';
constructor(
private readonly tenantDbService: TenantDatabaseService,
private readonly meilisearchService: MeilisearchService,
private readonly chunkerService: SemanticChunkerService,
private readonly semanticLinkService: SemanticLinkService,
) {}
async refreshRecord(
tenantId: string,
objectApiName: string,
recordId: string,
userId?: string,
trigger: string = 'manual',
) {
this.logger.log(
`Semantic refresh start: ${objectApiName}:${recordId} (trigger=${trigger})`,
);
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const objectDefinition = await knex('object_definitions').where({ apiName: objectApiName }).first();
if (!objectDefinition) {
this.logger.warn(`Object definition ${objectApiName} not found. Skipping semantic refresh.`);
return { skipped: true };
}
const tableName = this.getTableName(objectDefinition);
const record = await knex(tableName).where({ id: recordId }).first();
if (!record) {
this.logger.warn(`Record not found for semantic refresh: ${objectApiName}:${recordId}`);
return { skipped: true };
}
const comments = await knex('comments')
.where({
parent_object_api_name: objectApiName,
parent_record_id: recordId,
})
.orderBy('created_at', 'asc');
this.logger.log(
`Semantic refresh source: ${objectApiName}:${recordId} comments=${comments.length}`,
);
const adapter = this.adapters.find((candidate) => candidate.supports(objectApiName))!;
const projection = adapter.buildProjection({
objectApiName,
record,
objectDefinition,
comments,
});
const documentId = await this.upsertSemanticDocument(knex, projection);
const chunks = this.chunkerService.chunkText(projection.embeddingNarrative, comments);
this.logger.log(
`Semantic refresh chunking: ${objectApiName}:${recordId} chunks=${chunks.length}`,
);
await this.replaceChunks(knex, documentId, chunks);
const openAiConfig = await this.getOpenAiConfig(resolvedTenantId);
const embedderReady = await this.indexChunks(resolvedTenantId, projection, chunks, openAiConfig);
await this.generateSuggestions(
resolvedTenantId,
projection,
chunks,
openAiConfig,
embedderReady,
userId,
trigger,
);
this.logger.log(
`Semantic refresh complete: ${objectApiName}:${recordId} document=${documentId}`,
);
return { documentId, chunkCount: chunks.length };
}
async reindexObject(tenantId: string, objectApiName: string, userId?: string, limit = 250) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const objectDefinition = await knex('object_definitions').where({ apiName: objectApiName }).first();
if (!objectDefinition) {
return { total: 0, processed: 0 };
}
const tableName = this.getTableName(objectDefinition);
const records = await knex(tableName).select('id').limit(limit);
let processed = 0;
for (const record of records) {
await this.refreshRecord(resolvedTenantId, objectApiName, record.id, userId, 'batch_reindex');
processed += 1;
}
return { total: records.length, processed };
}
private async upsertSemanticDocument(knex: any, projection: any): Promise<string> {
const existing = await knex('semantic_documents')
.where({ entity_type: projection.entityType, entity_id: projection.entityId })
.first();
if (existing) {
await knex('semantic_documents')
.where({ id: existing.id })
.update({
title: projection.title,
narrative: projection.narrative,
metadata: JSON.stringify(projection.metadata || {}),
source_summary: JSON.stringify(projection.sourceSummary || {}),
updated_at: knex.fn.now(),
});
return existing.id;
}
const newId = randomUUID();
const [created] = await knex('semantic_documents')
.insert({
id: newId,
entity_type: projection.entityType,
entity_id: projection.entityId,
title: projection.title,
narrative: projection.narrative,
metadata: JSON.stringify(projection.metadata || {}),
source_summary: JSON.stringify(projection.sourceSummary || {}),
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
})
.returning('id');
if (created && typeof created === 'object' && created.id) {
return created.id;
}
// MySQL may return a numeric insert id (often 0 for UUID PKs). Always trust the generated UUID.
return newId;
}
private async replaceChunks(knex: any, documentId: string, chunks: any[]) {
if (!documentId) {
this.logger.warn('Skipping chunk replace: missing semantic document id.');
return;
}
await knex('semantic_chunks').where({ semantic_document_id: documentId }).delete();
if (!chunks.length) return;
await knex('semantic_chunks').insert(
chunks.map((chunk) => ({
semantic_document_id: documentId,
chunk_index: chunk.chunkIndex,
source_kind: chunk.sourceKind,
source_ref_id: chunk.sourceRefId,
text: chunk.text,
metadata: JSON.stringify(chunk.metadata || {}),
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
})),
);
}
private async indexChunks(
tenantId: string,
projection: any,
chunks: any[],
openAiConfig: OpenAIConfig | null,
) {
if (!this.meilisearchService.isEnabled()) {
this.logger.warn('Meilisearch disabled; skipping semantic chunk indexing.');
return false;
}
const indexName = this.meilisearchService.buildSemanticChunkIndexName(tenantId);
let embedderReady = false;
if (openAiConfig?.apiKey) {
embedderReady = await this.meilisearchService.ensureOpenAiEmbedder(indexName, {
embedderName: this.semanticEmbedderName,
apiKey: openAiConfig.apiKey,
model: openAiConfig.embeddingModel || this.defaultEmbeddingModel,
documentTemplate: '{{doc.title}}\n{{doc.text}}',
});
this.logger.log(
`Meilisearch embedder ensured: index=${indexName} model=${openAiConfig.embeddingModel || this.defaultEmbeddingModel}`,
);
} else {
this.logger.warn('OpenAI embedder not configured; semantic search will be lexical only.');
}
this.logger.log(`Indexing semantic chunks: index=${indexName} count=${chunks.length}`);
await this.meilisearchService.upsertDocuments(indexName, chunks.map((chunk) => ({
id: `${projection.entityType}_${projection.entityId}_${chunk.chunkIndex}`,
entityType: projection.entityType,
entityId: projection.entityId,
title: projection.title,
sourceKind: chunk.sourceKind,
sourceRefId: chunk.sourceRefId,
text: chunk.text,
})));
return embedderReady;
}
private async generateSuggestions(
tenantId: string,
projection: any,
chunks: any[],
openAiConfig: OpenAIConfig | null,
embedderReady: boolean,
userId?: string,
trigger: string = 'semantic_refresh',
) {
if (!this.meilisearchService.isEnabled() || !chunks.length) {
this.logger.warn(
`Skipping suggestion generation: meili=${this.meilisearchService.isEnabled()} chunks=${chunks.length}`,
);
return;
}
const indexName = this.meilisearchService.buildSemanticChunkIndexName(tenantId);
// Build query from all chunks (base record + comments), prioritising comments
// since they carry the most distinctive semantic signal.
const commentChunks = chunks.filter((c) => c.sourceKind === 'comment');
const baseChunks = chunks.filter((c) => c.sourceKind !== 'comment');
const orderedChunks = [...commentChunks, ...baseChunks];
const queryText = orderedChunks.map((chunk) => chunk.text).join(' ').slice(0, 1200);
this.logger.log(
`Generating suggestions: index=${indexName} queryLen=${queryText.length} hybrid=${embedderReady}`,
);
const search = await this.meilisearchService.searchIndex(
indexName,
queryText,
20,
// semanticRatio:1.0 = pure vector search, no lexical component that would
// match on shared tokens like 'name:' or 'Comment 1:' across all records.
embedderReady ? { embedder: this.semanticEmbedderName, semanticRatio: 1.0 } : undefined,
);
this.logger.log(
`Meilisearch results: index=${indexName} hits=${search.hits?.length || 0} total=${search.total}`,
);
const candidates = new Map<string, { hit: any; confidence: number; rankingDetails?: any }>();
for (const hit of search.hits || []) {
// Skip self
if (hit.entityId === projection.entityId) continue;
const confidence = hit._semanticScore ?? hit._rankingScore ?? 0;
// Use a lower threshold for comment chunks (short, conversational text
// naturally produces lower cosine similarity than structured field values).
const isComment = hit.sourceKind === 'comment';
const threshold = isComment ? this.MIN_CONFIDENCE_COMMENT : this.MIN_CONFIDENCE_BASE;
this.logger.log(
`Suggestion candidate: ${hit.entityType}:${hit.entityId} confidence=${confidence.toFixed(4)} kind=${hit.sourceKind || 'base'} threshold=${threshold} text="${String(hit.text || '').substring(0, 60)}"`,
);
if (confidence < threshold) {
this.logger.log(
`Skipping low-confidence match: ${hit.entityType}:${hit.entityId} confidence=${confidence.toFixed(4)} < ${threshold} (${isComment ? 'comment' : 'base'})`,
);
continue;
}
const key = `${hit.entityType}:${hit.entityId}`;
const existing = candidates.get(key);
if (!existing || confidence > existing.confidence) {
candidates.set(key, {
hit,
confidence,
rankingDetails: hit._rankingScoreDetails || null,
});
}
}
this.logger.log(`Filtered suggestions: ${candidates.size} passed thresholds (base=${this.MIN_CONFIDENCE_BASE}, comment=${this.MIN_CONFIDENCE_COMMENT})`);
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
for (const [key, { hit, confidence, rankingDetails }] of candidates.entries()) {
const [targetType, targetId] = key.split(':');
const llmAssessment = await this.assessLinkWithLlm(
openAiConfig,
trigger,
projection,
chunks,
hit,
confidence,
rankingDetails,
);
const reason =
llmAssessment?.reason ||
this.humanizeTrigger(trigger) ||
'Suggested from semantic similarity.';
await this.semanticLinkService.upsertSuggestedLink(knex, {
sourceEntityType: projection.entityType,
sourceEntityId: projection.entityId,
targetEntityType: targetType,
targetEntityId: targetId,
linkType: llmAssessment?.linkType || 'related',
status: 'suggested',
origin: 'semantic',
confidence,
reason,
evidence: this.buildEvidencePayload(
trigger,
chunks,
hit,
confidence,
rankingDetails,
llmAssessment,
),
suggestedByUserId: userId || null,
});
}
}
private buildEvidencePayload(
trigger: string,
chunks: any[],
hit: any,
confidence: number,
rankingDetails: any,
llmAssessment?: {
reason?: string;
explanation?: string;
matchedSignals?: string[];
} | null,
) {
return {
trigger,
explanation:
llmAssessment?.explanation ||
llmAssessment?.reason ||
'Suggested using semantic similarity and ranked chunk evidence.',
sourceSignals: chunks.slice(0, 2).map((chunk) => ({
sourceKind: chunk.sourceKind,
text: chunk.text.slice(0, 220),
})),
matchedSignals: llmAssessment?.matchedSignals || [],
matchedChunks: [
{
sourceKind: hit.sourceKind,
text: String(hit.text || '').slice(0, 220),
score: confidence,
rankingDetails: rankingDetails || null,
},
],
};
}
private async assessLinkWithLlm(
openAiConfig: OpenAIConfig | null,
trigger: string,
projection: any,
chunks: any[],
hit: any,
confidence: number,
rankingDetails: any,
): Promise<{ linkType: string; reason?: string; explanation?: string; matchedSignals?: string[] } | null> {
if (!openAiConfig?.apiKey) {
return null;
}
const promptPayload = {
trigger,
source: {
entityType: projection.entityType,
title: projection.title,
narrative: String(projection.narrative || '').slice(0, 900),
keySignals: chunks.slice(0, 3).map((chunk) => ({
sourceKind: chunk.sourceKind,
text: String(chunk.text || '').slice(0, 220),
})),
},
target: {
entityType: hit.entityType,
title: hit.title,
sourceKind: hit.sourceKind,
text: String(hit.text || '').slice(0, 300),
},
confidence,
rankingDetails: rankingDetails || {},
allowedLinkTypes: [
'related',
'supports',
'contradicts',
'expands',
'duplicate_of',
'references',
'depends_on',
],
};
try {
const model = new ChatOpenAI({
apiKey: openAiConfig.apiKey,
model: openAiConfig.model || this.defaultChatModel,
temperature: 0.1,
});
const response = await model.invoke([
new SystemMessage(
'Classify semantic relationship. Return valid JSON only with keys: linkType, reason, explanation, matchedSignals. linkType must be one of related|supports|contradicts|expands|duplicate_of|references|depends_on.',
),
new HumanMessage(JSON.stringify(promptPayload)),
]);
const content = typeof response.content === 'string'
? response.content
: Array.isArray(response.content)
? response.content.map((part: any) => (typeof part === 'string' ? part : part?.text || '')).join('')
: '';
const normalized = this.extractJsonObject(content);
if (!normalized) return null;
const linkType = this.normalizeLinkType(normalized.linkType);
return {
linkType,
reason: typeof normalized.reason === 'string' ? normalized.reason.trim() : undefined,
explanation:
typeof normalized.explanation === 'string' ? normalized.explanation.trim() : undefined,
matchedSignals: Array.isArray(normalized.matchedSignals)
? normalized.matchedSignals
.map((item: any) => String(item || '').trim())
.filter(Boolean)
.slice(0, 3)
: undefined,
};
} catch (error) {
this.logger.warn(`Semantic LLM assessment failed: ${error.message}`);
return null;
}
}
private extractJsonObject(raw: string): Record<string, any> | null {
if (!raw) return null;
const trimmed = raw.trim();
try {
return JSON.parse(trimmed);
} catch {
const match = trimmed.match(/\{[\s\S]*\}/);
if (!match) return null;
try {
return JSON.parse(match[0]);
} catch {
return null;
}
}
}
private normalizeLinkType(value: any): string {
const supported = new Set([
'related',
'supports',
'contradicts',
'expands',
'duplicate_of',
'references',
'depends_on',
]);
const normalized = String(value || '').trim().toLowerCase();
if (supported.has(normalized)) return normalized;
return 'related';
}
private humanizeTrigger(trigger: string): string {
if (!trigger) return 'Suggested from semantic similarity.';
const map: Record<string, string> = {
comment_created: 'Suggested based on a comment added to the record.',
comment_updated: 'Suggested based on a comment update.',
manual_refresh: 'Suggested after a manual semantic refresh.',
batch_reindex: 'Suggested during semantic reindexing.',
};
return map[trigger] || 'Suggested from semantic similarity.';
}
private getTableName(objectDefinition: any): string {
if (objectDefinition.tableName) return objectDefinition.tableName;
if (objectDefinition.pluralLabel) {
return objectDefinition.pluralLabel.toLowerCase().replace(/[^a-z0-9]+/g, '_');
}
return `${objectDefinition.apiName.toLowerCase()}s`;
}
private async getOpenAiConfig(tenantId: string): Promise<OpenAIConfig | null> {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const centralPrisma = getCentralPrisma();
const tenant = await centralPrisma.tenant.findUnique({
where: { id: resolvedTenantId },
select: { integrationsConfig: true },
});
let config = tenant?.integrationsConfig
? typeof tenant.integrationsConfig === 'string'
? this.tenantDbService.decryptIntegrationsConfig(tenant.integrationsConfig)
: tenant.integrationsConfig
: null;
if (!config?.openai && process.env.OPENAI_API_KEY) {
config = {
...(config || {}),
openai: {
apiKey: process.env.OPENAI_API_KEY,
embeddingModel: this.defaultEmbeddingModel,
},
};
}
if (config?.openai?.apiKey) {
return {
apiKey: config.openai.apiKey,
embeddingModel: config.openai.embeddingModel || this.defaultEmbeddingModel,
};
}
return null;
}
}

View File

@@ -1,42 +0,0 @@
import { Injectable, Logger } from '@nestjs/common';
import { InjectQueue } from '@nestjs/bullmq';
import { Queue } from 'bullmq';
import {
SEMANTIC_REFRESH_QUEUE,
SEMANTIC_REFRESH_JOB,
} from '../semantic-refresh.constants';
import { SemanticRefreshJobData } from '../semantic-refresh.processor';
@Injectable()
export class SemanticRefreshQueueService {
private readonly logger = new Logger(SemanticRefreshQueueService.name);
constructor(
@InjectQueue(SEMANTIC_REFRESH_QUEUE) private readonly queue: Queue,
) {}
async enqueue(
tenantId: string,
objectApiName: string,
recordId: string,
userId?: string,
trigger: string = 'manual',
): Promise<void> {
const data: SemanticRefreshJobData = {
tenantId,
objectApiName,
recordId,
userId,
trigger,
};
await this.queue.add(SEMANTIC_REFRESH_JOB, data, {
attempts: 3,
backoff: { type: 'exponential', delay: 2000 },
removeOnComplete: 100,
removeOnFail: 50,
});
this.logger.debug(
`Enqueued semantic refresh: ${objectApiName}:${recordId} trigger=${trigger}`,
);
}
}

View File

@@ -3,15 +3,13 @@ import {
FastifyAdapter,
NestFastifyApplication,
} from '@nestjs/platform-fastify';
import { ValidationPipe, Logger } from '@nestjs/common';
import { ValidationPipe } from '@nestjs/common';
import { AppModule } from './app.module';
import { VoiceService } from './voice/voice.service';
import { AudioConverterService } from './voice/audio-converter.service';
async function bootstrap() {
const app = await NestFactory.create<NestFastifyApplication>(
AppModule,
new FastifyAdapter({ logger: true }),
new FastifyAdapter(),
);
// Global validation pipe
@@ -35,145 +33,6 @@ async function bootstrap() {
const port = process.env.PORT || 3000;
await app.listen(port, '0.0.0.0');
// After app is listening, register WebSocket handler
const fastifyInstance = app.getHttpAdapter().getInstance();
const logger = new Logger('MediaStreamWS');
const voiceService = app.get(VoiceService);
const audioConverter = app.get(AudioConverterService);
const WebSocketServer = require('ws').Server;
const wss = new WebSocketServer({ noServer: true });
// Handle WebSocket upgrades at the server level
const server = (fastifyInstance.server as any);
// Track active Media Streams connections: streamSid -> WebSocket
const mediaStreams: Map<string, any> = new Map();
server.on('upgrade', (request: any, socket: any, head: any) => {
if (request.url === '/api/voice/media-stream') {
logger.log('=== MEDIA STREAM WEBSOCKET UPGRADE REQUEST ===');
logger.log(`Path: ${request.url}`);
wss.handleUpgrade(request, socket, head, (ws: any) => {
logger.log('=== MEDIA STREAM WEBSOCKET UPGRADED SUCCESSFULLY ===');
handleMediaStreamSocket(ws);
});
}
});
async function handleMediaStreamSocket(ws: any) {
let streamSid: string | null = null;
let callSid: string | null = null;
let tenantDomain: string | null = null;
let mediaPacketCount = 0;
ws.on('message', async (message: Buffer) => {
try {
const msg = JSON.parse(message.toString());
switch (msg.event) {
case 'connected':
logger.log('=== MEDIA STREAM EVENT: CONNECTED ===');
logger.log(`Protocol: ${msg.protocol}`);
logger.log(`Version: ${msg.version}`);
break;
case 'start':
streamSid = msg.streamSid;
callSid = msg.start.callSid;
tenantDomain = msg.start.customParameters?.tenantId || 'tenant1';
logger.log(`=== MEDIA STREAM EVENT: START ===`);
logger.log(`StreamSid: ${streamSid}`);
logger.log(`CallSid: ${callSid}`);
logger.log(`Tenant: ${tenantDomain}`);
logger.log(`MediaFormat: ${JSON.stringify(msg.start.mediaFormat)}`);
mediaStreams.set(streamSid, ws);
logger.log(`Stored WebSocket for streamSid: ${streamSid}. Total active streams: ${mediaStreams.size}`);
// Initialize OpenAI Realtime connection
logger.log(`Initializing OpenAI Realtime for call ${callSid}...`);
try {
await voiceService.initializeOpenAIRealtime({
callSid,
tenantId: tenantDomain,
userId: msg.start.customParameters?.userId || 'system',
});
logger.log(`✓ OpenAI Realtime initialized for call ${callSid}`);
} catch (error: any) {
logger.error(`Failed to initialize OpenAI: ${error.message}`);
}
break;
case 'media':
mediaPacketCount++;
// Only log every 500 packets to reduce noise
if (mediaPacketCount % 500 === 0) {
logger.log(`Received media packet #${mediaPacketCount} for StreamSid: ${streamSid}`);
}
if (!callSid || !tenantDomain) {
logger.warn('Received media before start event');
break;
}
try {
// Convert Twilio audio (μ-law 8kHz) to OpenAI format (PCM16 24kHz)
const twilioAudio = msg.media.payload;
const openaiAudio = audioConverter.twilioToOpenAI(twilioAudio);
// Send audio to OpenAI Realtime API
await voiceService.sendAudioToOpenAI(callSid, openaiAudio);
} catch (error: any) {
logger.error(`Error processing media: ${error.message}`);
}
break;
case 'stop':
logger.log(`=== MEDIA STREAM EVENT: STOP ===`);
logger.log(`StreamSid: ${streamSid}`);
logger.log(`Total media packets received: ${mediaPacketCount}`);
if (streamSid) {
mediaStreams.delete(streamSid);
logger.log(`Removed WebSocket for streamSid: ${streamSid}`);
}
// Clean up OpenAI connection
if (callSid) {
try {
logger.log(`Cleaning up OpenAI connection for call ${callSid}...`);
await voiceService.cleanupOpenAIConnection(callSid);
logger.log(`✓ OpenAI connection cleaned up`);
} catch (error: any) {
logger.error(`Failed to cleanup OpenAI: ${error.message}`);
}
}
break;
default:
logger.debug(`Unknown media stream event: ${msg.event}`);
}
} catch (error: any) {
logger.error(`Error processing media stream message: ${error.message}`);
}
});
ws.on('close', () => {
logger.log(`=== MEDIA STREAM WEBSOCKET CLOSED ===`);
if (streamSid) {
mediaStreams.delete(streamSid);
}
});
ws.on('error', (error: Error) => {
logger.error(`=== MEDIA STREAM WEBSOCKET ERROR ===`);
logger.error(`Error message: ${error.message}`);
});
}
console.log(`🚀 Application is running on: http://localhost:${port}/api`);
}

View File

@@ -1,5 +1,5 @@
import { Injectable, Logger } from '@nestjs/common';
import { Knex } from 'knex';
import type { Knex } from 'knex';
export interface CustomMigrationRecord {
id: string;

View File

@@ -1,38 +1,7 @@
import { Model, ModelOptions, QueryContext } from 'objection';
import { Model, ModelOptions, QueryContext, snakeCaseMappers } from 'objection';
export class BaseModel extends Model {
/**
* Use a minimal column mapper: keep property names as-is, but handle
* timestamp fields that are stored as created_at/updated_at in the DB.
*/
static columnNameMappers = {
parse(dbRow: Record<string, any>) {
const mapped: Record<string, any> = {};
for (const [key, value] of Object.entries(dbRow || {})) {
if (key === 'created_at') {
mapped.createdAt = value;
} else if (key === 'updated_at') {
mapped.updatedAt = value;
} else {
mapped[key] = value;
}
}
return mapped;
},
format(model: Record<string, any>) {
const mapped: Record<string, any> = {};
for (const [key, value] of Object.entries(model || {})) {
if (key === 'createdAt') {
mapped.created_at = value;
} else if (key === 'updatedAt') {
mapped.updated_at = value;
} else {
mapped[key] = value;
}
}
return mapped;
},
};
static columnNameMappers = snakeCaseMappers();
id: string;
createdAt: Date;

View File

@@ -1,33 +0,0 @@
import { BaseModel } from './base.model';
export class ContactDetail extends BaseModel {
static tableName = 'contact_details';
id!: string;
relatedObjectType!: 'Account' | 'Contact';
relatedObjectId!: string;
detailType!: string;
label?: string;
value!: string;
isPrimary!: boolean;
// Provide optional relations for each supported parent type.
static relationMappings = {
account: {
relation: BaseModel.BelongsToOneRelation,
modelClass: 'account.model',
join: {
from: 'contact_details.relatedObjectId',
to: 'accounts.id',
},
},
contact: {
relation: BaseModel.BelongsToOneRelation,
modelClass: 'contact.model',
join: {
from: 'contact_details.relatedObjectId',
to: 'contacts.id',
},
},
};
}

View File

@@ -1,30 +0,0 @@
import { BaseModel } from './base.model';
export class Contact extends BaseModel {
static tableName = 'contacts';
id!: string;
firstName!: string;
lastName!: string;
accountId!: string;
ownerId?: string;
static relationMappings = {
account: {
relation: BaseModel.BelongsToOneRelation,
modelClass: 'account.model',
join: {
from: 'contacts.accountId',
to: 'accounts.id',
},
},
owner: {
relation: BaseModel.BelongsToOneRelation,
modelClass: 'user.model',
join: {
from: 'contacts.ownerId',
to: 'users.id',
},
},
};
}

View File

@@ -30,8 +30,6 @@ export interface UIMetadata {
step?: number; // For number
accept?: string; // For file/image
relationDisplayField?: string; // Which field to display for relations
relationObjects?: string[]; // For polymorphic relations
relationTypeField?: string; // Field API name storing the selected relation type
// Formatting
format?: string; // Date format, number format, etc.
@@ -66,6 +64,9 @@ export class FieldDefinition extends BaseModel {
isCustom!: boolean;
displayOrder!: number;
uiMetadata?: UIMetadata;
// Field-level permissions
defaultReadable!: boolean;
defaultWritable!: boolean;
static relationMappings = {
objectDefinition: {
@@ -76,13 +77,5 @@ export class FieldDefinition extends BaseModel {
to: 'object_definitions.id',
},
},
rolePermissions: {
relation: BaseModel.HasManyRelation,
modelClass: () => require('./role-field-permission.model').RoleFieldPermission,
join: {
from: 'field_definitions.id',
to: 'role_field_permissions.fieldDefinitionId',
},
},
};
}

View File

@@ -10,11 +10,15 @@ export class ObjectDefinition extends BaseModel {
description?: string;
isSystem: boolean;
isCustom: boolean;
orgWideDefault: 'private' | 'public_read' | 'public_read_write';
// Authorization fields
accessModel: 'public' | 'owner' | 'mixed';
publicRead: boolean;
publicCreate: boolean;
publicUpdate: boolean;
publicDelete: boolean;
ownerField: string;
createdAt: Date;
updatedAt: Date;
fields?: any[];
rolePermissions?: any[];
static get jsonSchema() {
return {
@@ -28,14 +32,19 @@ export class ObjectDefinition extends BaseModel {
description: { type: 'string' },
isSystem: { type: 'boolean' },
isCustom: { type: 'boolean' },
orgWideDefault: { type: 'string', enum: ['private', 'public_read', 'public_read_write'] },
accessModel: { type: 'string', enum: ['public', 'owner', 'mixed'] },
publicRead: { type: 'boolean' },
publicCreate: { type: 'boolean' },
publicUpdate: { type: 'boolean' },
publicDelete: { type: 'boolean' },
ownerField: { type: 'string' },
},
};
}
static get relationMappings() {
const { FieldDefinition } = require('./field-definition.model');
const { RoleObjectPermission } = require('./role-object-permission.model');
const { RecordShare } = require('./record-share.model');
return {
fields: {
@@ -46,12 +55,12 @@ export class ObjectDefinition extends BaseModel {
to: 'field_definitions.objectDefinitionId',
},
},
rolePermissions: {
recordShares: {
relation: BaseModel.HasManyRelation,
modelClass: RoleObjectPermission,
modelClass: RecordShare,
join: {
from: 'object_definitions.id',
to: 'role_object_permissions.objectDefinitionId',
to: 'record_shares.objectDefinitionId',
},
},
};

View File

@@ -1,80 +1,39 @@
import { BaseModel } from './base.model';
export interface RecordShareAccessLevel {
canRead: boolean;
canEdit: boolean;
canDelete: boolean;
}
export class RecordShare extends BaseModel {
static tableName = 'record_shares';
// Don't use snake_case mapping since DB columns are already camelCase
static get columnNameMappers() {
return {
parse(obj: any) {
return obj;
},
format(obj: any) {
return obj;
},
};
}
// Don't auto-set timestamps - let DB defaults handle them
$beforeInsert() {
// Don't call super - skip BaseModel's timestamp logic
}
$beforeUpdate() {
// Don't call super - skip BaseModel's timestamp logic
}
id!: string;
objectDefinitionId!: string;
recordId!: string;
granteeUserId!: string;
grantedByUserId!: string;
accessLevel!: RecordShareAccessLevel;
actions!: any; // JSON field - will be string[] when parsed
fields?: any; // JSON field - will be string[] when parsed
expiresAt?: Date;
revokedAt?: Date;
createdAt!: Date;
updatedAt!: Date;
static get jsonSchema() {
return {
type: 'object',
required: ['objectDefinitionId', 'recordId', 'granteeUserId', 'grantedByUserId', 'accessLevel'],
required: ['objectDefinitionId', 'recordId', 'granteeUserId', 'grantedByUserId', 'actions'],
properties: {
id: { type: 'string' },
objectDefinitionId: { type: 'string' },
recordId: { type: 'string' },
granteeUserId: { type: 'string' },
grantedByUserId: { type: 'string' },
accessLevel: {
type: 'object',
properties: {
canRead: { type: 'boolean' },
canEdit: { type: 'boolean' },
canDelete: { type: 'boolean' },
actions: {
type: 'array',
items: { type: 'string' },
},
fields: {
type: ['array', 'null'],
items: { type: 'string' },
},
expiresAt: {
anyOf: [
{ type: 'string', format: 'date-time' },
{ type: 'null' },
{ type: 'object' } // Allow Date objects
]
},
revokedAt: {
anyOf: [
{ type: 'string', format: 'date-time' },
{ type: 'null' },
{ type: 'object' } // Allow Date objects
]
},
createdAt: { type: ['string', 'object'], format: 'date-time' },
updatedAt: { type: ['string', 'object'], format: 'date-time' },
expiresAt: { type: ['string', 'null'], format: 'date-time' },
revokedAt: { type: ['string', 'null'], format: 'date-time' },
},
};
}
@@ -110,4 +69,11 @@ export class RecordShare extends BaseModel {
},
};
}
// Check if share is currently valid
isValid(): boolean {
if (this.revokedAt) return false;
if (this.expiresAt && new Date(this.expiresAt) < new Date()) return false;
return true;
}
}

View File

@@ -1,51 +0,0 @@
import { BaseModel } from './base.model';
export class RoleFieldPermission extends BaseModel {
static tableName = 'role_field_permissions';
id!: string;
roleId!: string;
fieldDefinitionId!: string;
canRead!: boolean;
canEdit!: boolean;
createdAt!: Date;
updatedAt!: Date;
static get jsonSchema() {
return {
type: 'object',
required: ['roleId', 'fieldDefinitionId'],
properties: {
id: { type: 'string' },
roleId: { type: 'string' },
fieldDefinitionId: { type: 'string' },
canRead: { type: 'boolean' },
canEdit: { type: 'boolean' },
},
};
}
static get relationMappings() {
const { Role } = require('./role.model');
const { FieldDefinition } = require('./field-definition.model');
return {
role: {
relation: BaseModel.BelongsToOneRelation,
modelClass: Role,
join: {
from: 'role_field_permissions.roleId',
to: 'roles.id',
},
},
fieldDefinition: {
relation: BaseModel.BelongsToOneRelation,
modelClass: FieldDefinition,
join: {
from: 'role_field_permissions.fieldDefinitionId',
to: 'field_definitions.id',
},
},
};
}
}

View File

@@ -1,59 +0,0 @@
import { BaseModel } from './base.model';
export class RoleObjectPermission extends BaseModel {
static tableName = 'role_object_permissions';
id!: string;
roleId!: string;
objectDefinitionId!: string;
canCreate!: boolean;
canRead!: boolean;
canEdit!: boolean;
canDelete!: boolean;
canViewAll!: boolean;
canModifyAll!: boolean;
createdAt!: Date;
updatedAt!: Date;
static get jsonSchema() {
return {
type: 'object',
required: ['roleId', 'objectDefinitionId'],
properties: {
id: { type: 'string' },
roleId: { type: 'string' },
objectDefinitionId: { type: 'string' },
canCreate: { type: 'boolean' },
canRead: { type: 'boolean' },
canEdit: { type: 'boolean' },
canDelete: { type: 'boolean' },
canViewAll: { type: 'boolean' },
canModifyAll: { type: 'boolean' },
},
};
}
static get relationMappings() {
const { Role } = require('./role.model');
const { ObjectDefinition } = require('./object-definition.model');
return {
role: {
relation: BaseModel.BelongsToOneRelation,
modelClass: Role,
join: {
from: 'role_object_permissions.roleId',
to: 'roles.id',
},
},
objectDefinition: {
relation: BaseModel.BelongsToOneRelation,
modelClass: ObjectDefinition,
join: {
from: 'role_object_permissions.objectDefinitionId',
to: 'object_definitions.id',
},
},
};
}
}

View File

@@ -0,0 +1,38 @@
import { BaseModel } from './base.model';
export class RoleRule extends BaseModel {
static tableName = 'role_rules';
id: string;
roleId: string;
rulesJson: any[]; // Array of CASL rules
createdAt: Date;
updatedAt: Date;
static get jsonSchema() {
return {
type: 'object',
required: ['roleId', 'rulesJson'],
properties: {
id: { type: 'string' },
roleId: { type: 'string' },
rulesJson: { type: 'array' },
},
};
}
static get relationMappings() {
const { Role } = require('./role.model');
return {
role: {
relation: BaseModel.BelongsToOneRelation,
modelClass: Role,
join: {
from: 'role_rules.roleId',
to: 'roles.id',
},
},
};
}
}

View File

@@ -27,8 +27,7 @@ export class Role extends BaseModel {
const { RolePermission } = require('./role-permission.model');
const { Permission } = require('./permission.model');
const { User } = require('./user.model');
const { RoleObjectPermission } = require('./role-object-permission.model');
const { RoleFieldPermission } = require('./role-field-permission.model');
const { RoleRule } = require('./role-rule.model');
return {
rolePermissions: {
@@ -63,20 +62,12 @@ export class Role extends BaseModel {
to: 'users.id',
},
},
objectPermissions: {
roleRules: {
relation: BaseModel.HasManyRelation,
modelClass: RoleObjectPermission,
modelClass: RoleRule,
join: {
from: 'roles.id',
to: 'role_object_permissions.roleId',
},
},
fieldPermissions: {
relation: BaseModel.HasManyRelation,
modelClass: RoleFieldPermission,
join: {
from: 'roles.id',
to: 'role_field_permissions.roleId',
to: 'role_rules.roleId',
},
},
};

View File

@@ -1,5 +1,4 @@
import { BaseModel } from './base.model';
import { ModelOptions, QueryContext } from 'objection';
export class User extends BaseModel {
static tableName = 'users';
@@ -9,8 +8,6 @@ export class User extends BaseModel {
password: string;
firstName?: string;
lastName?: string;
alias?: string;
name?: string;
isActive: boolean;
createdAt: Date;
updatedAt: Date;
@@ -25,40 +22,15 @@ export class User extends BaseModel {
password: { type: 'string' },
firstName: { type: 'string' },
lastName: { type: 'string' },
alias: { type: 'string' },
name: { type: 'string' },
isActive: { type: 'boolean' },
},
};
}
/**
* Compute the `name` column before insert/update so lookup fields
* referencing User.name always have a value.
*/
private computeName() {
if (this.alias) {
this.name = this.alias;
} else if (this.firstName || this.lastName) {
this.name = [this.firstName, this.lastName].filter(Boolean).join(' ');
} else if (this.email) {
this.name = this.email;
}
}
$beforeInsert(queryContext: QueryContext) {
super.$beforeInsert(queryContext);
this.computeName();
}
$beforeUpdate(opt: ModelOptions, queryContext: QueryContext) {
super.$beforeUpdate(opt, queryContext);
this.computeName();
}
static get relationMappings() {
const { UserRole } = require('./user-role.model');
const { Role } = require('./role.model');
const { RecordShare } = require('./record-share.model');
return {
userRoles: {
@@ -81,6 +53,22 @@ export class User extends BaseModel {
to: 'roles.id',
},
},
sharesGranted: {
relation: BaseModel.HasManyRelation,
modelClass: RecordShare,
join: {
from: 'users.id',
to: 'record_shares.grantedByUserId',
},
},
sharesReceived: {
relation: BaseModel.HasManyRelation,
modelClass: RecordShare,
join: {
from: 'users.id',
to: 'record_shares.granteeUserId',
},
},
};
}
}

View File

@@ -22,9 +22,7 @@ export interface FieldConfigDTO {
step?: number;
accept?: string;
relationObject?: string;
relationObjects?: string[];
relationDisplayField?: string;
relationTypeField?: string;
format?: string;
prefix?: string;
suffix?: string;
@@ -45,14 +43,6 @@ export interface ObjectDefinitionDTO {
description?: string;
isSystem: boolean;
fields: FieldConfigDTO[];
relatedLists?: Array<{
title: string;
relationName: string;
objectApiName: string;
fields: FieldConfigDTO[];
canCreate?: boolean;
createRoute?: string;
}>;
}
@Injectable()
@@ -79,10 +69,6 @@ export class FieldMapperService {
const frontendType = this.mapFieldType(field.type);
const isLookupField = frontendType === 'belongsTo' || field.type.toLowerCase().includes('lookup');
// Hide 'id' field from list view by default
const isIdField = field.apiName === 'id';
const defaultShowOnList = isIdField ? false : true;
return {
id: field.id,
apiName: field.apiName,
@@ -99,7 +85,7 @@ export class FieldMapperService {
isReadOnly: field.isSystem || uiMetadata.isReadOnly || false,
// View visibility
showOnList: uiMetadata.showOnList !== undefined ? uiMetadata.showOnList : defaultShowOnList,
showOnList: uiMetadata.showOnList !== false,
showOnDetail: uiMetadata.showOnDetail !== false,
showOnEdit: uiMetadata.showOnEdit !== false && !field.isSystem,
sortable: uiMetadata.sortable !== false,
@@ -112,12 +98,10 @@ export class FieldMapperService {
step: uiMetadata.step,
accept: uiMetadata.accept,
relationObject: field.referenceObject,
relationObjects: uiMetadata.relationObjects,
// For lookup fields, provide default display field if not specified
relationDisplayField: isLookupField
? (uiMetadata.relationDisplayField || 'name')
: uiMetadata.relationDisplayField,
relationTypeField: uiMetadata.relationTypeField,
// Formatting
format: uiMetadata.format,
@@ -145,14 +129,12 @@ export class FieldMapperService {
'boolean': 'boolean',
'date': 'date',
'datetime': 'datetime',
'date_time': 'datetime',
'time': 'time',
'email': 'email',
'url': 'url',
'phone': 'text',
'picklist': 'select',
'multipicklist': 'multiSelect',
'multi_picklist': 'multiSelect',
'lookup': 'belongsTo',
'master-detail': 'belongsTo',
'currency': 'currency',
@@ -224,17 +206,6 @@ export class FieldMapperService {
.filter((f: any) => f.isActive !== false)
.sort((a: any, b: any) => (a.displayOrder || 0) - (b.displayOrder || 0))
.map((f: any) => this.mapFieldToDTO(f)),
relatedLists: (objectDef.relatedLists || []).map((list: any) => ({
title: list.title,
relationName: list.relationName,
objectApiName: list.objectApiName,
fields: (list.fields || [])
.filter((f: any) => f.isActive !== false)
.map((f: any) => this.mapFieldToDTO(f))
.filter((f: any) => f.showOnList !== false),
canCreate: list.canCreate,
createRoute: list.createRoute,
})),
};
}

View File

@@ -1,5 +1,4 @@
import { Model } from 'objection';
import { randomUUID } from 'crypto';
/**
* Base model for all dynamic and system models
@@ -11,23 +10,26 @@ export class BaseModel extends Model {
tenantId?: string;
ownerId?: string;
name?: string;
created_at?: string;
updated_at?: string;
created_at?: Date;
updated_at?: Date;
// Hook to set system-managed fields
async $beforeInsert() {
if (!this.id) {
this.id = randomUUID();
}
if (!this.created_at) {
this.created_at = new Date().toISOString().slice(0, 19).replace('T', ' ');
}
if (!this.updated_at) {
this.updated_at = new Date().toISOString().slice(0, 19).replace('T', ' ');
}
$beforeInsert() {
// created_at and updated_at are handled by the database
// ownerId should be set by the controller/service
}
async $beforeUpdate() {
this.updated_at = new Date().toISOString().slice(0, 19).replace('T', ' ');
$beforeUpdate() {
// updated_at is handled by the database
}
/**
* Get the API name for this object
* Override in subclasses
*/
static get objectApiName(): string {
return 'BaseModel';
}
}

View File

@@ -1,3 +1,4 @@
import { randomUUID } from 'crypto';
import { ModelClass, JSONSchema, RelationMappings, Model } from 'objection';
import { BaseModel } from './base.model';
@@ -27,14 +28,6 @@ export interface ObjectMetadata {
}
export class DynamicModelFactory {
/**
* Get relation name from lookup field API name
* Converts "ownerId" -> "owner", "customFieldId" -> "customfield"
*/
static getRelationName(lookupFieldApiName: string): string {
return lookupFieldApiName.replace(/Id$/, '').toLowerCase();
}
/**
* Create a dynamic model class from object metadata
* @param meta Object metadata
@@ -56,7 +49,8 @@ export class DynamicModelFactory {
updated_at: { type: 'string', format: 'date-time' },
};
// Don't require id or tenantId - they'll be set automatically
// Don't require system-managed fields (id, tenantId, ownerId, timestamps)
// These are auto-set by hooks or database
const required: string[] = [];
// Add custom fields
@@ -76,13 +70,20 @@ export class DynamicModelFactory {
// Store lookup fields metadata for later use
const lookupFieldsInfo = lookupFields.map(f => ({
apiName: f.apiName,
relationName: DynamicModelFactory.getRelationName(f.apiName),
relationName: f.apiName.replace(/Id$/, '').toLowerCase(),
referenceObject: f.referenceObject,
targetTable: this.getTableName(f.referenceObject),
}));
// Create the dynamic model class extending BaseModel
class DynamicModel extends BaseModel {
// Create the dynamic model class extending Model directly
class DynamicModel extends Model {
id?: string;
tenantId?: string;
ownerId?: string;
name?: string;
created_at?: string;
updated_at?: string;
static tableName = tableName;
static objectApiName = apiName;
@@ -119,47 +120,6 @@ export class DynamicModelFactory {
};
}
// Add additional relation mappings (e.g., hasMany)
for (const relation of relations) {
if (mappings[relation.name]) {
continue;
}
let modelClass: any = relation.targetObjectApiName;
if (getModel) {
const resolvedModel = getModel(relation.targetObjectApiName);
if (resolvedModel) {
modelClass = resolvedModel;
} else {
continue;
}
}
const targetTable = DynamicModelFactory.getTableName(relation.targetObjectApiName);
if (relation.type === 'belongsTo') {
mappings[relation.name] = {
relation: Model.BelongsToOneRelation,
modelClass,
join: {
from: `${tableName}.${relation.fromColumn}`,
to: `${targetTable}.${relation.toColumn}`,
},
};
}
if (relation.type === 'hasMany') {
mappings[relation.name] = {
relation: Model.HasManyRelation,
modelClass,
join: {
from: `${tableName}.${relation.fromColumn}`,
to: `${targetTable}.${relation.toColumn}`,
},
};
}
}
return mappings;
}
@@ -170,6 +130,23 @@ export class DynamicModelFactory {
properties,
};
}
async $beforeInsert() {
if (!this.id) {
this.id = randomUUID();
}
if (!this.created_at) {
this.created_at = new Date().toISOString().slice(0, 19).replace('T', ' ');
}
if (!this.updated_at) {
this.updated_at = new Date().toISOString().slice(0, 19).replace('T', ' ');
}
}
async $beforeUpdate(opt: any, queryContext: any) {
await super.$beforeUpdate(opt, queryContext);
this.updated_at = new Date().toISOString().slice(0, 19).replace('T', ' ');
}
}
return DynamicModel as any;
@@ -179,7 +156,6 @@ export class DynamicModelFactory {
* Convert a field definition to JSON schema property
*/
private static fieldToJsonSchema(field: FieldDefinition): Record<string, any> {
const baseSchema = () => {
switch (field.type.toUpperCase()) {
case 'TEXT':
case 'STRING':
@@ -227,18 +203,6 @@ export class DynamicModelFactory {
default:
return { type: 'string' };
}
};
const schema = baseSchema();
// Allow null for non-required fields so optional strings/numbers don't fail validation
if (!field.isRequired) {
return {
anyOf: [schema, { type: 'null' }],
};
}
return schema;
}
/**
@@ -250,9 +214,6 @@ export class DynamicModelFactory {
.replace(/([A-Z])/g, '_$1')
.toLowerCase()
.replace(/^_/, '');
if (snakeCase.endsWith('y')) {
return `${snakeCase.slice(0, -1)}ies`;
}
return snakeCase.endsWith('s') ? snakeCase : `${snakeCase}s`;
}
}

View File

@@ -16,17 +16,13 @@ export class ModelRegistry {
*/
registerModel(apiName: string, modelClass: ModelClass<BaseModel>): void {
this.registry.set(apiName, modelClass);
const lowerKey = apiName.toLowerCase();
if (lowerKey !== apiName && !this.registry.has(lowerKey)) {
this.registry.set(lowerKey, modelClass);
}
}
/**
* Get a model from the registry
*/
getModel(apiName: string): ModelClass<BaseModel> {
const model = this.registry.get(apiName) || this.registry.get(apiName.toLowerCase());
const model = this.registry.get(apiName);
if (!model) {
throw new Error(`Model for ${apiName} not found in registry`);
}
@@ -37,7 +33,7 @@ export class ModelRegistry {
* Check if a model exists in the registry
*/
hasModel(apiName: string): boolean {
return this.registry.has(apiName) || this.registry.has(apiName.toLowerCase());
return this.registry.has(apiName);
}
/**
@@ -50,8 +46,7 @@ export class ModelRegistry {
// Returns undefined if model not found (for models not yet registered)
const model = DynamicModelFactory.createModel(
metadata,
(apiName: string) =>
this.registry.get(apiName) || this.registry.get(apiName.toLowerCase()),
(apiName: string) => this.registry.get(apiName),
);
this.registerModel(metadata.apiName, model);
return model;

View File

@@ -1,5 +1,5 @@
import { Injectable, Logger } from '@nestjs/common';
import { Knex } from 'knex';
import type { Knex } from 'knex';
import { ModelClass } from 'objection';
import { BaseModel } from './base.model';
import { ModelRegistry } from './model.registry';
@@ -171,25 +171,6 @@ export class ModelService {
}
}
if (objectMetadata.relations) {
for (const relation of objectMetadata.relations) {
if (relation.targetObjectApiName) {
try {
await this.ensureModelWithDependencies(
tenantId,
relation.targetObjectApiName,
fetchMetadata,
visited,
);
} catch (error) {
this.logger.debug(
`Skipping registration of related model ${relation.targetObjectApiName}: ${error.message}`
);
}
}
}
}
// Now create and register this model (all dependencies are ready)
await this.createModelForObject(tenantId, objectMetadata);
this.logger.log(`Registered model for ${objectApiName} in tenant ${tenantId}`);

View File

@@ -6,14 +6,11 @@ import { SchemaManagementService } from './schema-management.service';
import { FieldMapperService } from './field-mapper.service';
import { TenantModule } from '../tenant/tenant.module';
import { MigrationModule } from '../migration/migration.module';
import { RbacModule } from '../rbac/rbac.module';
import { ModelRegistry } from './models/model.registry';
import { ModelService } from './models/model.service';
import { MeilisearchModule } from '../search/meilisearch.module';
import { KnowledgeModule } from '../knowledge/knowledge.module';
@Module({
imports: [TenantModule, MigrationModule, RbacModule, MeilisearchModule, KnowledgeModule],
imports: [TenantModule, MigrationModule],
providers: [
ObjectService,
SchemaManagementService,

File diff suppressed because it is too large Load Diff

View File

@@ -95,49 +95,4 @@ export class RuntimeObjectController {
user.userId,
);
}
@Post(':objectApiName/records/bulk-delete')
async deleteRecords(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Body() body: { recordIds?: string[]; ids?: string[] },
@CurrentUser() user: any,
) {
const recordIds: string[] = body?.recordIds || body?.ids || [];
return this.objectService.deleteRecords(
tenantId,
objectApiName,
recordIds,
user.userId,
);
}
/**
* Direct filter-based search — used when applying a saved list view.
* Bypasses the AI planning step; accepts pre-resolved structured filters.
*/
@Post(':objectApiName/records/search')
async searchRecords(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@CurrentUser() user: any,
@Body() body: {
filters?: Array<{ field: string; operator: string; value?: any; values?: any[]; from?: string; to?: string }>;
sort?: { field: string; direction: 'asc' | 'desc' } | null;
page?: number;
pageSize?: number;
},
) {
const page = Number.isFinite(Number(body?.page)) ? Number(body.page) : 1;
const pageSize = Number.isFinite(Number(body?.pageSize)) ? Number(body.pageSize) : 25;
return this.objectService.searchRecordsWithFilters(
tenantId,
objectApiName,
user.userId,
body?.filters || [],
{ page, pageSize },
body?.sort || undefined,
);
}
}

View File

@@ -1,5 +1,5 @@
import { Injectable, Logger } from '@nestjs/common';
import { Knex } from 'knex';
import type { Knex } from 'knex';
import { ObjectDefinition } from '../models/object-definition.model';
import { FieldDefinition } from '../models/field-definition.model';
@@ -15,11 +15,7 @@ export class SchemaManagementService {
objectDefinition: ObjectDefinition,
fields: FieldDefinition[],
) {
const tableName = this.getTableName(
objectDefinition.apiName,
objectDefinition.label,
objectDefinition.pluralLabel,
);
const tableName = this.getTableName(objectDefinition.apiName);
// Check if table already exists
const exists = await knex.schema.hasTable(tableName);
@@ -48,10 +44,8 @@ export class SchemaManagementService {
knex: Knex,
objectApiName: string,
field: FieldDefinition,
objectLabel?: string,
pluralLabel?: string,
) {
const tableName = this.getTableName(objectApiName, objectLabel, pluralLabel);
const tableName = this.getTableName(objectApiName);
await knex.schema.alterTable(tableName, (table) => {
this.addFieldColumn(table, field);
@@ -67,10 +61,8 @@ export class SchemaManagementService {
knex: Knex,
objectApiName: string,
fieldApiName: string,
objectLabel?: string,
pluralLabel?: string,
) {
const tableName = this.getTableName(objectApiName, objectLabel, pluralLabel);
const tableName = this.getTableName(objectApiName);
await knex.schema.alterTable(tableName, (table) => {
table.dropColumn(fieldApiName);
@@ -79,44 +71,11 @@ export class SchemaManagementService {
this.logger.log(`Removed field ${fieldApiName} from table ${tableName}`);
}
/**
* Alter a field in an existing object table
* Handles safe updates like changing NOT NULL or constraints
* Warns about potentially destructive operations
*/
async alterFieldInTable(
knex: Knex,
objectApiName: string,
fieldApiName: string,
field: FieldDefinition,
objectLabel?: string,
pluralLabel?: string,
options?: {
skipTypeChange?: boolean; // Skip if type change would lose data
},
) {
const tableName = this.getTableName(objectApiName, objectLabel, pluralLabel);
const skipTypeChange = options?.skipTypeChange ?? true;
await knex.schema.alterTable(tableName, (table) => {
// Drop the existing column and recreate with new definition
// Note: This approach works for metadata changes, but type changes may need data migration
table.dropColumn(fieldApiName);
});
// Recreate the column with new definition
await knex.schema.alterTable(tableName, (table) => {
this.addFieldColumn(table, field);
});
this.logger.log(`Altered field ${fieldApiName} in table ${tableName}`);
}
/**
* Drop an object table
*/
async dropObjectTable(knex: Knex, objectApiName: string, objectLabel?: string, pluralLabel?: string) {
const tableName = this.getTableName(objectApiName, objectLabel, pluralLabel);
async dropObjectTable(knex: Knex, objectApiName: string) {
const tableName = this.getTableName(objectApiName);
await knex.schema.dropTableIfExists(tableName);
@@ -135,30 +94,15 @@ export class SchemaManagementService {
let column: Knex.ColumnBuilder;
switch (field.type) {
// Text types
case 'String':
case 'TEXT':
case 'EMAIL':
case 'PHONE':
case 'URL':
column = table.string(columnName, field.length || 255);
break;
case 'Text':
case 'LONG_TEXT':
column = table.text(columnName);
break;
case 'PICKLIST':
case 'MULTI_PICKLIST':
column = table.string(columnName, 255);
break;
// Numeric types
case 'Number':
case 'NUMBER':
case 'CURRENCY':
case 'PERCENT':
if (field.scale && field.scale > 0) {
column = table.decimal(
columnName,
@@ -171,28 +115,18 @@ export class SchemaManagementService {
break;
case 'Boolean':
case 'BOOLEAN':
column = table.boolean(columnName).defaultTo(false);
break;
// Date types
case 'Date':
case 'DATE':
column = table.date(columnName);
break;
case 'DateTime':
case 'DATE_TIME':
column = table.datetime(columnName);
break;
case 'TIME':
column = table.time(columnName);
break;
// Relationship types
case 'Reference':
case 'LOOKUP':
column = table.uuid(columnName);
if (field.referenceObject) {
const refTableName = this.getTableName(field.referenceObject);
@@ -200,30 +134,19 @@ export class SchemaManagementService {
}
break;
// Email (legacy)
case 'Email':
column = table.string(columnName, 255);
break;
// Phone (legacy)
case 'Phone':
column = table.string(columnName, 50);
break;
// Url (legacy)
case 'Url':
column = table.string(columnName, 255);
break;
// File types
case 'FILE':
case 'IMAGE':
column = table.text(columnName); // Store file path or URL
break;
// JSON
case 'Json':
case 'JSON':
column = table.json(columnName);
break;
@@ -251,35 +174,16 @@ export class SchemaManagementService {
/**
* Convert object API name to table name (convert to snake_case, pluralize)
*/
private getTableName(apiName: string, objectLabel?: string, pluralLabel?: string): string {
const toSnakePlural = (source: string): string => {
const cleaned = source.replace(/[\s-]+/g, '_');
const snake = cleaned
.replace(/([a-z0-9])([A-Z])/g, '$1_$2')
.replace(/__+/g, '_')
private getTableName(apiName: string): string {
// Convert PascalCase to snake_case
const snakeCase = apiName
.replace(/([A-Z])/g, '_$1')
.toLowerCase()
.replace(/^_/, '');
if (snake.endsWith('y')) return `${snake.slice(0, -1)}ies`;
if (snake.endsWith('s')) return snake;
return `${snake}s`;
};
const fromApi = toSnakePlural(apiName);
const fromLabel = objectLabel ? toSnakePlural(objectLabel) : null;
const fromPlural = pluralLabel ? toSnakePlural(pluralLabel) : null;
if (fromLabel && fromLabel.includes('_') && !fromApi.includes('_')) {
return fromLabel;
}
if (fromPlural && fromPlural.includes('_') && !fromApi.includes('_')) {
return fromPlural;
}
if (fromLabel && fromLabel !== fromApi) return fromLabel;
if (fromPlural && fromPlural !== fromApi) return fromPlural;
return fromApi;
// Simple pluralization (append 's' if not already plural)
// In production, use a proper pluralization library
return snakeCase.endsWith('s') ? snakeCase : `${snakeCase}s`;
}
/**

View File

@@ -2,18 +2,19 @@ import {
Controller,
Get,
Post,
Patch,
Put,
Delete,
Param,
Body,
UseGuards,
Inject,
} from '@nestjs/common';
import { ObjectService } from './object.service';
import { FieldMapperService } from './field-mapper.service';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { TenantId } from '../tenant/tenant.decorator';
import { TenantDatabaseService } from '../tenant/tenant-database.service';
import { ObjectDefinition } from '../models/object-definition.model';
import { FieldDefinition } from '../models/field-definition.model';
import { Knex } from 'knex';
@Controller('setup/objects')
@UseGuards(JwtAuthGuard)
@@ -21,7 +22,7 @@ export class SetupObjectController {
constructor(
private objectService: ObjectService,
private fieldMapperService: FieldMapperService,
private tenantDbService: TenantDatabaseService,
@Inject('KnexConnection') private readonly knex: Knex,
) {}
@Get()
@@ -73,84 +74,121 @@ export class SetupObjectController {
return this.fieldMapperService.mapFieldToDTO(field);
}
@Put(':objectApiName/fields/:fieldApiName')
async updateFieldDefinition(
// Access & Permissions endpoints
/**
* Get object access configuration
*/
@Get(':objectApiName/access')
async getAccess(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Param('fieldApiName') fieldApiName: string,
@Body() data: any,
) {
const field = await this.objectService.updateFieldDefinition(
tenantId,
objectApiName,
fieldApiName,
data,
);
return this.fieldMapperService.mapFieldToDTO(field);
const objectDef = await ObjectDefinition.query(this.knex)
.findOne({ apiName: objectApiName })
.withGraphFetched('fields');
if (!objectDef) {
throw new Error('Object definition not found');
}
@Delete(':objectApiName/fields/:fieldApiName')
async deleteFieldDefinition(
return {
accessModel: objectDef.accessModel,
publicRead: objectDef.publicRead,
publicCreate: objectDef.publicCreate,
publicUpdate: objectDef.publicUpdate,
publicDelete: objectDef.publicDelete,
ownerField: objectDef.ownerField,
fields: objectDef['fields'] || [],
};
}
/**
* Update object access configuration
*/
@Put(':objectApiName/access')
async updateAccess(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Param('fieldApiName') fieldApiName: string,
@Body() dto: any,
) {
return this.objectService.deleteFieldDefinition(
tenantId,
objectApiName,
fieldApiName,
);
console.log('dto', JSON.stringify(dto));
const objectDef = await ObjectDefinition.query(this.knex)
.findOne({ apiName: objectApiName });
if (!objectDef) {
throw new Error('Object definition not found');
}
@Patch(':objectApiName')
async updateObjectDefinition(
return ObjectDefinition.query(this.knex).patchAndFetchById(objectDef.id, dto);
}
/**
* Create or update field-level permissions
*/
@Post(':objectApiName/fields/:fieldKey/permissions')
async setFieldPermissions(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Body() data: any,
@Param('fieldKey') fieldKey: string,
@Body() dto: any,
) {
return this.objectService.updateObjectDefinition(tenantId, objectApiName, data);
const objectDef = await ObjectDefinition.query(this.knex)
.findOne({ apiName: objectApiName });
if (!objectDef) {
throw new Error('Object definition not found');
}
@Get(':objectId/field-permissions')
async getFieldPermissions(
@TenantId() tenantId: string,
@Param('objectId') objectId: string,
) {
return this.objectService.getFieldPermissions(tenantId, objectId);
// Find the field definition
const field = await FieldDefinition.query(this.knex)
.findOne({
objectDefinitionId: objectDef.id,
apiName: fieldKey,
});
if (!field) {
throw new Error('Field definition not found');
}
@Put(':objectId/field-permissions')
async updateFieldPermission(
@TenantId() tenantId: string,
@Param('objectId') objectId: string,
@Body() data: { roleId: string; fieldDefinitionId: string; canRead: boolean; canEdit: boolean },
) {
return this.objectService.updateFieldPermission(tenantId, data.roleId, data.fieldDefinitionId, data.canRead, data.canEdit);
// Update field permissions
return FieldDefinition.query(this.knex).patchAndFetchById(field.id, {
defaultReadable: dto.defaultReadable ?? field.defaultReadable,
defaultWritable: dto.defaultWritable ?? field.defaultWritable,
});
}
@Get(':objectApiName/permissions/:roleId')
async getObjectPermissions(
/**
* Bulk set field permissions for an object
*/
@Put(':objectApiName/field-permissions')
async bulkSetFieldPermissions(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Param('roleId') roleId: string,
@Body() fields: { fieldKey: string; defaultReadable: boolean; defaultWritable: boolean }[],
) {
return this.objectService.getObjectPermissions(tenantId, objectApiName, roleId);
const objectDef = await ObjectDefinition.query(this.knex)
.findOne({ apiName: objectApiName });
if (!objectDef) {
throw new Error('Object definition not found');
}
@Put(':objectApiName/permissions')
async updateObjectPermissions(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Body() data: {
roleId: string;
canCreate: boolean;
canRead: boolean;
canEdit: boolean;
canDelete: boolean;
canViewAll: boolean;
canModifyAll: boolean;
},
) {
return this.objectService.updateObjectPermissions(tenantId, objectApiName, data);
// Update each field in the field_definitions table
for (const fieldUpdate of fields) {
await FieldDefinition.query(this.knex)
.where({
objectDefinitionId: objectDef.id,
apiName: fieldUpdate.fieldKey,
})
.patch({
defaultReadable: fieldUpdate.defaultReadable,
defaultWritable: fieldUpdate.defaultWritable,
});
}
return { success: true };
}
}

View File

@@ -1,6 +1,4 @@
import { IsString, IsUUID, IsBoolean, IsOptional, IsObject, IsIn } from 'class-validator';
export type PageLayoutType = 'detail' | 'list';
import { IsString, IsUUID, IsBoolean, IsOptional, IsObject } from 'class-validator';
export class CreatePageLayoutDto {
@IsString()
@@ -9,27 +7,19 @@ export class CreatePageLayoutDto {
@IsUUID()
objectId: string;
@IsIn(['detail', 'list'])
@IsOptional()
layoutType?: PageLayoutType = 'detail';
@IsBoolean()
@IsOptional()
isDefault?: boolean;
@IsObject()
layoutConfig: {
// For detail layouts: grid-based field positions
fields: Array<{
fieldId: string;
x?: number;
y?: number;
w?: number;
h?: number;
// For list layouts: field order (optional, defaults to array index)
order?: number;
x: number;
y: number;
w: number;
h: number;
}>;
relatedLists?: string[];
};
@IsString()
@@ -51,13 +41,11 @@ export class UpdatePageLayoutDto {
layoutConfig?: {
fields: Array<{
fieldId: string;
x?: number;
y?: number;
w?: number;
h?: number;
order?: number;
x: number;
y: number;
w: number;
h: number;
}>;
relatedLists?: string[];
};
@IsString()

View File

@@ -10,7 +10,7 @@ import {
Query,
} from '@nestjs/common';
import { PageLayoutService } from './page-layout.service';
import { CreatePageLayoutDto, UpdatePageLayoutDto, PageLayoutType } from './dto/page-layout.dto';
import { CreatePageLayoutDto, UpdatePageLayoutDto } from './dto/page-layout.dto';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { TenantId } from '../tenant/tenant.decorator';
@@ -25,21 +25,13 @@ export class PageLayoutController {
}
@Get()
findAll(
@TenantId() tenantId: string,
@Query('objectId') objectId?: string,
@Query('layoutType') layoutType?: PageLayoutType,
) {
return this.pageLayoutService.findAll(tenantId, objectId, layoutType);
findAll(@TenantId() tenantId: string, @Query('objectId') objectId?: string) {
return this.pageLayoutService.findAll(tenantId, objectId);
}
@Get('default/:objectId')
findDefaultByObject(
@TenantId() tenantId: string,
@Param('objectId') objectId: string,
@Query('layoutType') layoutType?: PageLayoutType,
) {
return this.pageLayoutService.findDefaultByObject(tenantId, objectId, layoutType || 'detail');
findDefaultByObject(@TenantId() tenantId: string, @Param('objectId') objectId: string) {
return this.pageLayoutService.findDefaultByObject(tenantId, objectId);
}
@Get(':id')

View File

@@ -1,26 +1,24 @@
import { Injectable, NotFoundException } from '@nestjs/common';
import { TenantDatabaseService } from '../tenant/tenant-database.service';
import { CreatePageLayoutDto, UpdatePageLayoutDto, PageLayoutType } from './dto/page-layout.dto';
import { CreatePageLayoutDto, UpdatePageLayoutDto } from './dto/page-layout.dto';
@Injectable()
export class PageLayoutService {
constructor(private tenantDbService: TenantDatabaseService) {}
async create(tenantId: string, createDto: CreatePageLayoutDto) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const layoutType = createDto.layoutType || 'detail';
const knex = await this.tenantDbService.getTenantKnex(tenantId);
// If this layout is set as default, unset other defaults for the same object and layout type
// If this layout is set as default, unset other defaults for the same object
if (createDto.isDefault) {
await knex('page_layouts')
.where({ object_id: createDto.objectId, layout_type: layoutType })
.where({ object_id: createDto.objectId })
.update({ is_default: false });
}
const [id] = await knex('page_layouts').insert({
name: createDto.name,
object_id: createDto.objectId,
layout_type: layoutType,
is_default: createDto.isDefault || false,
layout_config: JSON.stringify(createDto.layoutConfig),
description: createDto.description || null,
@@ -31,8 +29,8 @@ export class PageLayoutService {
return result;
}
async findAll(tenantId: string, objectId?: string, layoutType?: PageLayoutType) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
async findAll(tenantId: string, objectId?: string) {
const knex = await this.tenantDbService.getTenantKnex(tenantId);
let query = knex('page_layouts');
@@ -40,16 +38,12 @@ export class PageLayoutService {
query = query.where({ object_id: objectId });
}
if (layoutType) {
query = query.where({ layout_type: layoutType });
}
const layouts = await query.orderByRaw('is_default DESC, name ASC');
return layouts;
}
async findOne(tenantId: string, id: string) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
const layout = await knex('page_layouts').where({ id }).first();
@@ -60,26 +54,27 @@ export class PageLayoutService {
return layout;
}
async findDefaultByObject(tenantId: string, objectId: string, layoutType: PageLayoutType = 'detail') {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
async findDefaultByObject(tenantId: string, objectId: string) {
const knex = await this.tenantDbService.getTenantKnex(tenantId);
const layout = await knex('page_layouts')
.where({ object_id: objectId, is_default: true, layout_type: layoutType })
.where({ object_id: objectId, is_default: true })
.first();
return layout || null;
}
async update(tenantId: string, id: string, updateDto: UpdatePageLayoutDto) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
// Check if layout exists
const layout = await this.findOne(tenantId, id);
await this.findOne(tenantId, id);
// If setting as default, unset other defaults for the same object and layout type
// If setting as default, unset other defaults for the same object
if (updateDto.isDefault) {
const layout = await this.findOne(tenantId, id);
await knex('page_layouts')
.where({ object_id: layout.object_id, layout_type: layout.layout_type })
.where({ object_id: layout.object_id })
.whereNot({ id })
.update({ is_default: false });
}
@@ -112,7 +107,7 @@ export class PageLayoutService {
}
async remove(tenantId: string, id: string) {
const knex = await this.tenantDbService.getTenantKnexById(tenantId);
const knex = await this.tenantDbService.getTenantKnex(tenantId);
await this.findOne(tenantId, id);

View File

@@ -1,199 +0,0 @@
import { AbilityBuilder, PureAbility, AbilityClass } from '@casl/ability';
import { Injectable } from '@nestjs/common';
import { User } from '../models/user.model';
import { RoleObjectPermission } from '../models/role-object-permission.model';
import { RoleFieldPermission } from '../models/role-field-permission.model';
import { RecordShare } from '../models/record-share.model';
// Define action types
export type Action = 'create' | 'read' | 'update' | 'delete' | 'view_all' | 'modify_all';
// Define subject types - can be string (object API name) or actual object with fields
export type Subject = string | { objectApiName: string; ownerId?: string; id?: string; [key: string]: any };
// Define field actions
export type FieldAction = 'read' | 'edit';
export type AppAbility = PureAbility<[Action, Subject], { field?: string }>;
@Injectable()
export class AbilityFactory {
/**
* Build CASL ability for a user based on their roles and permissions
* This aggregates permissions from all roles the user has
*/
async defineAbilityFor(
user: User & { roles?: Array<{ objectPermissions?: RoleObjectPermission[]; fieldPermissions?: RoleFieldPermission[] }> },
recordShares?: RecordShare[],
): Promise<AppAbility> {
const { can, cannot, build } = new AbilityBuilder<AppAbility>(PureAbility as AbilityClass<AppAbility>);
if (!user.roles || user.roles.length === 0) {
// No roles = no permissions
return build();
}
// Aggregate object permissions from all roles
const objectPermissionsMap = new Map<string, {
canCreate: boolean;
canRead: boolean;
canEdit: boolean;
canDelete: boolean;
canViewAll: boolean;
canModifyAll: boolean;
}>();
// Aggregate field permissions from all roles
const fieldPermissionsMap = new Map<string, {
canRead: boolean;
canEdit: boolean;
}>();
// Process all roles
for (const role of user.roles) {
// Aggregate object permissions
if (role.objectPermissions) {
for (const perm of role.objectPermissions) {
const existing = objectPermissionsMap.get(perm.objectDefinitionId) || {
canCreate: false,
canRead: false,
canEdit: false,
canDelete: false,
canViewAll: false,
canModifyAll: false,
};
// Union of permissions (if any role grants it, user has it)
objectPermissionsMap.set(perm.objectDefinitionId, {
canCreate: existing.canCreate || perm.canCreate,
canRead: existing.canRead || perm.canRead,
canEdit: existing.canEdit || perm.canEdit,
canDelete: existing.canDelete || perm.canDelete,
canViewAll: existing.canViewAll || perm.canViewAll,
canModifyAll: existing.canModifyAll || perm.canModifyAll,
});
}
}
// Aggregate field permissions
if (role.fieldPermissions) {
for (const perm of role.fieldPermissions) {
const existing = fieldPermissionsMap.get(perm.fieldDefinitionId) || {
canRead: false,
canEdit: false,
};
fieldPermissionsMap.set(perm.fieldDefinitionId, {
canRead: existing.canRead || perm.canRead,
canEdit: existing.canEdit || perm.canEdit,
});
}
}
}
// Convert aggregated permissions to CASL rules
for (const [objectId, perms] of objectPermissionsMap) {
// Create permission
if (perms.canCreate) {
can('create', objectId);
}
// Read permission
if (perms.canRead) {
can('read', objectId);
}
// View all permission (can see all records regardless of ownership)
if (perms.canViewAll) {
can('view_all', objectId);
}
// Edit permission
if (perms.canEdit) {
can('update', objectId);
}
// Modify all permission (can edit all records regardless of ownership)
if (perms.canModifyAll) {
can('modify_all', objectId);
}
// Delete permission
if (perms.canDelete) {
can('delete', objectId);
}
}
// Add record sharing permissions
if (recordShares) {
for (const share of recordShares) {
// Only add if share is active (not expired, not revoked)
const now = new Date();
const isExpired = share.expiresAt && share.expiresAt < now;
const isRevoked = share.revokedAt !== null;
if (!isExpired && !isRevoked) {
// Note: Record-level sharing will be checked in authorization service
// CASL abilities are primarily for object-level permissions
// Individual record access is validated in applyScopeToQuery
}
}
}
return build();
}
/**
* Check if user can access a specific field
* Returns true if user has permission or if no restriction exists
*/
canAccessField(
fieldDefinitionId: string,
action: FieldAction,
user: User & { roles?: Array<{ fieldPermissions?: RoleFieldPermission[] }> },
): boolean {
if (!user.roles || user.roles.length === 0) {
return false;
}
// Collect all field permissions from all roles
const allFieldPermissions: RoleFieldPermission[] = [];
for (const role of user.roles) {
if (role.fieldPermissions) {
allFieldPermissions.push(...role.fieldPermissions);
}
}
// If there are NO field permissions configured at all, allow by default
if (allFieldPermissions.length === 0) {
return true;
}
// If field permissions exist, check for explicit grants (union of all roles)
for (const role of user.roles) {
if (role.fieldPermissions) {
const fieldPerm = role.fieldPermissions.find(fp => fp.fieldDefinitionId === fieldDefinitionId);
if (fieldPerm) {
if (action === 'read' && fieldPerm.canRead) return true;
if (action === 'edit' && fieldPerm.canEdit) return true;
}
}
}
// No explicit rule for this field but other field permissions exist.
// Default to allow so new fields don't get silently stripped and fail validation.
return true;
}
/**
* Filter fields based on user permissions
* Returns array of field IDs the user can access with the specified action
*/
filterFields(
fieldDefinitionIds: string[],
action: FieldAction,
user: User & { roles?: Array<{ fieldPermissions?: RoleFieldPermission[] }> },
): string[] {
return fieldDefinitionIds.filter(fieldId => this.canAccessField(fieldId, action, user));
}
}

View File

@@ -1,282 +0,0 @@
import { Injectable, ForbiddenException } from '@nestjs/common';
import { Knex } from 'knex';
import { User } from '../models/user.model';
import { ObjectDefinition } from '../models/object-definition.model';
import { FieldDefinition } from '../models/field-definition.model';
import { RecordShare } from '../models/record-share.model';
import { AbilityFactory, AppAbility, Action } from './ability.factory';
import { DynamicModelFactory } from '../object/models/dynamic-model.factory';
import { subject } from '@casl/ability';
@Injectable()
export class AuthorizationService {
constructor(private abilityFactory: AbilityFactory) {}
/**
* Apply authorization scope to a query based on OWD and user permissions
* This determines which records the user can see
* Modifies the query in place and returns void
*/
async applyScopeToQuery<T = any>(
query: any, // Accept both Knex and Objection query builders
objectDef: ObjectDefinition,
user: User & { roles?: any[] },
action: Action,
knex: Knex,
): Promise<void> {
// Get user's ability
const recordShares = await this.getActiveRecordShares(objectDef.id, user.id, knex);
const ability = await this.abilityFactory.defineAbilityFor(user, recordShares);
// Check if user has the base permission for this action
// Use object ID, not API name, since permissions are stored by object ID
if (!ability.can(action, objectDef.id)) {
// No permission at all - return empty result
query.where(knex.raw('1 = 0'));
return;
}
// Check special permissions
const hasViewAll = ability.can('view_all', objectDef.id);
const hasModifyAll = ability.can('modify_all', objectDef.id);
// If user has view_all or modify_all, they can see all records
if (hasViewAll || hasModifyAll) {
// No filtering needed
return;
}
// Apply OWD (Org-Wide Default) restrictions
switch (objectDef.orgWideDefault) {
case 'public_read_write':
// Everyone can see all records
return;
case 'public_read':
// Everyone can see all records (write operations checked separately)
return;
case 'private':
default:
// Only owner and explicitly shared records
await this.applyPrivateScope(query, objectDef, user, recordShares, knex);
return;
}
}
/**
* Apply private scope: owner + shared records
*/
private async applyPrivateScope<T = any>(
query: any, // Accept both Knex and Objection query builders
objectDef: ObjectDefinition,
user: User,
recordShares: RecordShare[],
knex: Knex,
): Promise<void> {
const tableName = this.getTableName(objectDef.apiName);
// Check if table has ownerId column
const hasOwner = await knex.schema.hasColumn(tableName, 'ownerId');
if (!hasOwner && recordShares.length === 0) {
// No ownership and no shares - user can't see anything
query.where(knex.raw('1 = 0'));
return;
}
// Build conditions: ownerId = user OR record shared with user
query.where((builder) => {
if (hasOwner) {
builder.orWhere(`${tableName}.ownerId`, user.id);
}
if (recordShares.length > 0) {
const sharedRecordIds = recordShares.map(share => share.recordId);
builder.orWhereIn(`${tableName}.id`, sharedRecordIds);
}
});
}
/**
* Check if user can perform action on a specific record
*/
async canPerformAction(
action: Action,
objectDef: ObjectDefinition,
record: any,
user: User & { roles?: any[] },
knex: Knex,
): Promise<boolean> {
const recordShares = await this.getActiveRecordShares(objectDef.id, user.id, knex);
const ability = await this.abilityFactory.defineAbilityFor(user, recordShares);
// Check base permission - use object ID not API name
if (!ability.can(action, objectDef.id)) {
return false;
}
// Check special permissions - use object ID not API name
const hasViewAll = ability.can('view_all', objectDef.id);
const hasModifyAll = ability.can('modify_all', objectDef.id);
// canViewAll only grants read access to all records
if (action === 'read' && hasViewAll) {
return true;
}
// canModifyAll grants edit/delete access to all records
if ((action === 'update' || action === 'delete') && hasModifyAll) {
return true;
}
// Check OWD
switch (objectDef.orgWideDefault) {
case 'public_read_write':
return true;
case 'public_read':
if (action === 'read') return true;
// For write actions, check ownership
return record.ownerId === user.id;
case 'private':
default:
// Check ownership
if (record.ownerId === user.id) return true;
// Check if record is shared with user
const share = recordShares.find(s => s.recordId === record.id);
if (share) {
if (action === 'read' && share.accessLevel.canRead) return true;
if (action === 'update' && share.accessLevel.canEdit) return true;
if (action === 'delete' && share.accessLevel.canDelete) return true;
}
return false;
}
}
/**
* Filter data based on field-level permissions
* Removes fields the user cannot read
*/
async filterReadableFields(
data: any,
fields: FieldDefinition[],
user: User & { roles?: any[] },
): Promise<any> {
const filtered: any = {};
// Always include id - it's required for navigation and record identification
if (data.id !== undefined) {
filtered.id = data.id;
}
for (const field of fields) {
if (this.abilityFactory.canAccessField(field.id, 'read', user)) {
if (data[field.apiName] !== undefined) {
filtered[field.apiName] = data[field.apiName];
}
// For lookup fields, also include the related object (e.g., ownerId -> owner)
if (field.type === 'LOOKUP') {
const relationName = DynamicModelFactory.getRelationName(field.apiName);
if (data[relationName] !== undefined) {
filtered[relationName] = data[relationName];
}
}
}
}
return filtered;
}
/**
* Filter data based on field-level permissions
* Removes fields the user cannot edit
*/
async filterEditableFields(
data: any,
fields: FieldDefinition[],
user: User & { roles?: any[] },
): Promise<any> {
const filtered: any = {};
for (const field of fields) {
if (this.abilityFactory.canAccessField(field.id, 'edit', user)) {
if (data[field.apiName] !== undefined) {
filtered[field.apiName] = data[field.apiName];
}
}
}
return filtered;
}
/**
* Get active record shares for a user on an object
*/
private async getActiveRecordShares(
objectDefinitionId: string,
userId: string,
knex: Knex,
): Promise<RecordShare[]> {
const now = new Date();
return await RecordShare.query(knex)
.where('objectDefinitionId', objectDefinitionId)
.where('granteeUserId', userId)
.whereNull('revokedAt')
.where((builder) => {
builder.whereNull('expiresAt').orWhere('expiresAt', '>', now);
});
}
/**
* Check if user has permission to create records
*/
async canCreate(
objectDef: ObjectDefinition,
user: User & { roles?: any[] },
): Promise<boolean> {
const ability = await this.abilityFactory.defineAbilityFor(user, []);
return ability.can('create', objectDef.id);
}
/**
* Throw exception if user cannot perform action
*/
async assertCanPerformAction(
action: Action,
objectDef: ObjectDefinition,
record: any,
user: User & { roles?: any[] },
knex: Knex,
): Promise<void> {
const can = await this.canPerformAction(action, objectDef, record, user, knex);
if (!can) {
throw new ForbiddenException(`You do not have permission to ${action} this record`);
}
}
/**
* Get table name from API name
*/
private getTableName(apiName: string): string {
// Convert CamelCase to snake_case and pluralize
const snakeCase = apiName
.replace(/([A-Z])/g, '_$1')
.toLowerCase()
.replace(/^_/, '');
// Simple pluralization
if (snakeCase.endsWith('y')) {
return snakeCase.slice(0, -1) + 'ies';
} else if (snakeCase.endsWith('s')) {
return snakeCase;
} else {
return snakeCase + 's';
}
}
}

View File

@@ -1,19 +0,0 @@
import { IsString, IsBoolean, IsOptional, IsDateString } from 'class-validator';
export class CreateRecordShareDto {
@IsString()
granteeUserId: string;
@IsBoolean()
canRead: boolean;
@IsBoolean()
canEdit: boolean;
@IsBoolean()
canDelete: boolean;
@IsOptional()
@IsDateString()
expiresAt?: string;
}

View File

@@ -1,16 +1,14 @@
import { Module } from '@nestjs/common';
import { RbacService } from './rbac.service';
import { AbilityFactory } from './ability.factory';
import { AuthorizationService } from './authorization.service';
import { SetupRolesController } from './setup-roles.controller';
import { SetupUsersController } from './setup-users.controller';
import { RecordSharingController } from './record-sharing.controller';
import { ShareController } from './share.controller';
import { RoleController, RoleRuleController } from './role.controller';
import { UserController } from './user.controller';
import { TenantModule } from '../tenant/tenant.module';
@Module({
imports: [TenantModule],
controllers: [SetupRolesController, SetupUsersController, RecordSharingController],
providers: [RbacService, AbilityFactory, AuthorizationService],
exports: [RbacService, AbilityFactory, AuthorizationService],
providers: [RbacService],
controllers: [ShareController, RoleController, RoleRuleController, UserController],
exports: [RbacService],
})
export class RbacModule {}

View File

@@ -1,350 +0,0 @@
import {
Controller,
Get,
Post,
Delete,
Param,
Body,
UseGuards,
ForbiddenException,
} from '@nestjs/common';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { TenantId } from '../tenant/tenant.decorator';
import { CurrentUser } from '../auth/current-user.decorator';
import { TenantDatabaseService } from '../tenant/tenant-database.service';
import { RecordShare } from '../models/record-share.model';
import { ObjectDefinition } from '../models/object-definition.model';
import { User } from '../models/user.model';
import { AuthorizationService } from './authorization.service';
import { CreateRecordShareDto } from './dto/create-record-share.dto';
@Controller('runtime/objects/:objectApiName/records/:recordId/shares')
@UseGuards(JwtAuthGuard)
export class RecordSharingController {
constructor(
private tenantDbService: TenantDatabaseService,
private authService: AuthorizationService,
) {}
@Get()
async getRecordShares(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Param('recordId') recordId: string,
@CurrentUser() currentUser: any,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Get object definition
const objectDef = await ObjectDefinition.query(knex)
.findOne({ apiName: objectApiName });
if (!objectDef) {
throw new Error('Object not found');
}
// Get the record to check ownership
const tableName = this.getTableName(
objectDef.apiName,
objectDef.label,
objectDef.pluralLabel,
);
const record = await knex(tableName)
.where({ id: recordId })
.first();
if (!record) {
throw new Error('Record not found');
}
// Only owner can view shares
if (record.ownerId !== currentUser.userId) {
// Check if user has modify all permission
const user: any = await User.query(knex)
.findById(currentUser.userId)
.withGraphFetched('roles.objectPermissions');
if (!user) {
throw new ForbiddenException('User not found');
}
const hasModifyAll = user.roles?.some(role =>
role.objectPermissions?.some(
perm => perm.objectDefinitionId === objectDef.id && perm.canModifyAll
)
);
if (!hasModifyAll) {
throw new ForbiddenException('Only the record owner or users with Modify All permission can view shares');
}
}
// Get all active shares for this record
const shares = await RecordShare.query(knex)
.where({ objectDefinitionId: objectDef.id, recordId })
.whereNull('revokedAt')
.where(builder => {
builder.whereNull('expiresAt').orWhere('expiresAt', '>', new Date());
})
.withGraphFetched('[granteeUser]')
.orderBy('createdAt', 'desc');
return shares;
}
@Post()
async createRecordShare(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Param('recordId') recordId: string,
@CurrentUser() currentUser: any,
@Body() data: CreateRecordShareDto,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Get object definition
const objectDef = await ObjectDefinition.query(knex)
.findOne({ apiName: objectApiName });
if (!objectDef) {
throw new Error('Object not found');
}
// Get the record to check ownership
const tableName = this.getTableName(
objectDef.apiName,
objectDef.label,
objectDef.pluralLabel,
);
const record = await knex(tableName)
.where({ id: recordId })
.first();
if (!record) {
throw new Error('Record not found');
}
// Check if user can share - either owner or has modify permissions
const canShare = await this.canUserShareRecord(
currentUser.userId,
record,
objectDef,
knex,
);
if (!canShare) {
throw new ForbiddenException('You do not have permission to share this record');
}
// Cannot share with self
if (data.granteeUserId === currentUser.userId) {
throw new Error('Cannot share record with yourself');
}
// Check if share already exists
const existingShare = await RecordShare.query(knex)
.where({
objectDefinitionId: objectDef.id,
recordId,
granteeUserId: data.granteeUserId,
})
.whereNull('revokedAt')
.first();
if (existingShare) {
// Update existing share
const updated = await RecordShare.query(knex)
.patchAndFetchById(existingShare.id, {
accessLevel: {
canRead: data.canRead,
canEdit: data.canEdit,
canDelete: data.canDelete,
},
// Convert ISO string to MySQL datetime format
expiresAt: data.expiresAt
? knex.raw('?', [new Date(data.expiresAt).toISOString().slice(0, 19).replace('T', ' ')])
: null,
} as any);
return RecordShare.query(knex)
.findById(updated.id)
.withGraphFetched('[granteeUser]');
}
// Create new share
const share = await RecordShare.query(knex).insertAndFetch({
objectDefinitionId: objectDef.id,
recordId,
granteeUserId: data.granteeUserId,
grantedByUserId: currentUser.userId,
accessLevel: {
canRead: data.canRead,
canEdit: data.canEdit,
canDelete: data.canDelete,
},
// Convert ISO string to MySQL datetime format: YYYY-MM-DD HH:MM:SS
expiresAt: data.expiresAt
? knex.raw('?', [new Date(data.expiresAt).toISOString().slice(0, 19).replace('T', ' ')])
: null,
} as any);
return RecordShare.query(knex)
.findById(share.id)
.withGraphFetched('[granteeUser]');
}
@Delete(':shareId')
async deleteRecordShare(
@TenantId() tenantId: string,
@Param('objectApiName') objectApiName: string,
@Param('recordId') recordId: string,
@Param('shareId') shareId: string,
@CurrentUser() currentUser: any,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Get object definition
const objectDef = await ObjectDefinition.query(knex)
.findOne({ apiName: objectApiName });
if (!objectDef) {
throw new Error('Object not found');
}
// Get the record to check ownership
const tableName = this.getTableName(
objectDef.apiName,
objectDef.label,
objectDef.pluralLabel,
);
const record = await knex(tableName)
.where({ id: recordId })
.first();
if (!record) {
throw new Error('Record not found');
}
// Only owner can revoke shares
if (record.ownerId !== currentUser.userId) {
// Check if user has modify all permission
const user: any = await User.query(knex)
.findById(currentUser.userId)
.withGraphFetched('roles.objectPermissions');
if (!user) {
throw new ForbiddenException('User not found');
}
const hasModifyAll = user.roles?.some(role =>
role.objectPermissions?.some(
perm => perm.objectDefinitionId === objectDef.id && perm.canModifyAll
)
);
if (!hasModifyAll) {
throw new ForbiddenException('Only the record owner or users with Modify All permission can revoke shares');
}
}
// Revoke the share (soft delete)
await RecordShare.query(knex)
.patchAndFetchById(shareId, {
revokedAt: knex.fn.now() as any,
});
return { success: true };
}
private async canUserShareRecord(
userId: string,
record: any,
objectDef: ObjectDefinition,
knex: any,
): Promise<boolean> {
// Owner can always share
if (record.ownerId === userId) {
return true;
}
// Check if user has modify all or edit permissions
const user: any = await User.query(knex)
.findById(userId)
.withGraphFetched('roles.objectPermissions');
if (!user) {
return false;
}
// Check for canModifyAll permission
const hasModifyAll = user.roles?.some(role =>
role.objectPermissions?.some(
perm => perm.objectDefinitionId === objectDef.id && perm.canModifyAll
)
);
if (hasModifyAll) {
return true;
}
// Check for canEdit permission (user needs edit to share)
const hasEdit = user.roles?.some(role =>
role.objectPermissions?.some(
perm => perm.objectDefinitionId === objectDef.id && perm.canEdit
)
);
// If user has edit permission, check if they can actually edit this record
// by using the authorization service
if (hasEdit) {
try {
await this.authService.assertCanPerformAction(
'update',
objectDef,
record,
user,
knex,
);
return true;
} catch {
return false;
}
}
return false;
}
private getTableName(apiName: string, objectLabel?: string, pluralLabel?: string): string {
const toSnakePlural = (source: string): string => {
const cleaned = source.replace(/[\s-]+/g, '_');
const snake = cleaned
.replace(/([a-z0-9])([A-Z])/g, '$1_$2')
.replace(/__+/g, '_')
.toLowerCase()
.replace(/^_/, '');
if (snake.endsWith('y')) return `${snake.slice(0, -1)}ies`;
if (snake.endsWith('s')) return snake;
return `${snake}s`;
};
const fromApi = toSnakePlural(apiName);
const fromLabel = objectLabel ? toSnakePlural(objectLabel) : null;
const fromPlural = pluralLabel ? toSnakePlural(pluralLabel) : null;
if (fromLabel && fromLabel.includes('_') && !fromApi.includes('_')) {
return fromLabel;
}
if (fromPlural && fromPlural.includes('_') && !fromApi.includes('_')) {
return fromPlural;
}
if (fromLabel && fromLabel !== fromApi) return fromLabel;
if (fromPlural && fromPlural !== fromApi) return fromPlural;
return fromApi;
}
}

View File

@@ -0,0 +1,137 @@
import {
Controller,
Get,
Post,
Put,
Delete,
Body,
Param,
UseGuards,
Inject,
} from '@nestjs/common';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { Role } from '../models/role.model';
import { RoleRule } from '../models/role-rule.model';
import { Knex } from 'knex';
export class CreateRoleDto {
name: string;
guardName?: string;
description?: string;
}
export class UpdateRoleDto {
name?: string;
description?: string;
}
export class CreateRoleRuleDto {
roleId: string;
rulesJson: any[]; // Array of CASL rules
}
export class UpdateRoleRuleDto {
rulesJson: any[];
}
@Controller('roles')
@UseGuards(JwtAuthGuard)
export class RoleController {
constructor(@Inject('KnexConnection') private readonly knex: Knex) {}
/**
* List all roles
*/
@Get()
async list() {
return Role.query(this.knex).withGraphFetched('[roleRules]');
}
/**
* Get a single role by ID
*/
@Get(':id')
async get(@Param('id') id: string) {
return Role.query(this.knex)
.findById(id)
.withGraphFetched('[roleRules, permissions]');
}
/**
* Create a new role
*/
@Post()
async create(@Body() createDto: CreateRoleDto) {
return Role.query(this.knex).insert({
name: createDto.name,
guardName: createDto.guardName || 'api',
description: createDto.description,
});
}
/**
* Update a role
*/
@Put(':id')
async update(@Param('id') id: string, @Body() updateDto: UpdateRoleDto) {
return Role.query(this.knex).patchAndFetchById(id, updateDto);
}
/**
* Delete a role
*/
@Delete(':id')
async delete(@Param('id') id: string) {
await Role.query(this.knex).deleteById(id);
return { success: true };
}
}
@Controller('role-rules')
@UseGuards(JwtAuthGuard)
export class RoleRuleController {
constructor(@Inject('KnexConnection') private readonly knex: Knex) {}
/**
* Get rules for a role
*/
@Get('role/:roleId')
async getForRole(@Param('roleId') roleId: string) {
return RoleRule.query(this.knex).where('roleId', roleId);
}
/**
* Create or update role rules
* This will replace existing rules for the role
*/
@Post()
async createOrUpdate(@Body() dto: CreateRoleRuleDto) {
// Delete existing rules for this role
await RoleRule.query(this.knex).where('roleId', dto.roleId).delete();
// Insert new rules
return RoleRule.query(this.knex).insert({
roleId: dto.roleId,
rulesJson: dto.rulesJson,
});
}
/**
* Update role rules by ID
*/
@Put(':id')
async update(@Param('id') id: string, @Body() dto: UpdateRoleRuleDto) {
return RoleRule.query(this.knex).patchAndFetchById(id, {
rulesJson: dto.rulesJson,
});
}
/**
* Delete role rules
*/
@Delete(':id')
async delete(@Param('id') id: string) {
await RoleRule.query(this.knex).deleteById(id);
return { success: true };
}
}

View File

@@ -1,141 +0,0 @@
import {
Controller,
Get,
Post,
Patch,
Delete,
Param,
Body,
UseGuards,
} from '@nestjs/common';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { TenantId } from '../tenant/tenant.decorator';
import { TenantDatabaseService } from '../tenant/tenant-database.service';
import { Role } from '../models/role.model';
@Controller('setup/roles')
@UseGuards(JwtAuthGuard)
export class SetupRolesController {
constructor(private tenantDbService: TenantDatabaseService) {}
@Get()
async getRoles(@TenantId() tenantId: string) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
return await Role.query(knex).select('*').orderBy('name', 'asc');
}
@Get(':id')
async getRole(
@TenantId() tenantId: string,
@Param('id') id: string,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
return await Role.query(knex).findById(id).withGraphFetched('users');
}
@Post()
async createRole(
@TenantId() tenantId: string,
@Body() data: { name: string; description?: string; guardName?: string },
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const role = await Role.query(knex).insert({
name: data.name,
description: data.description,
guardName: data.guardName || 'tenant',
});
return role;
}
@Patch(':id')
async updateRole(
@TenantId() tenantId: string,
@Param('id') id: string,
@Body() data: { name?: string; description?: string; guardName?: string },
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const updateData: any = {};
if (data.name) updateData.name = data.name;
if (data.description !== undefined) updateData.description = data.description;
if (data.guardName) updateData.guardName = data.guardName;
const role = await Role.query(knex).patchAndFetchById(id, updateData);
return role;
}
@Delete(':id')
async deleteRole(
@TenantId() tenantId: string,
@Param('id') id: string,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Delete role user assignments first
await knex('user_roles').where({ roleId: id }).delete();
// Delete role permissions
await knex('role_permissions').where({ roleId: id }).delete();
await knex('role_object_permissions').where({ roleId: id }).delete();
// Delete the role
await Role.query(knex).deleteById(id);
return { success: true };
}
@Post(':roleId/users')
async addUserToRole(
@TenantId() tenantId: string,
@Param('roleId') roleId: string,
@Body() data: { userId: string },
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Check if assignment already exists
const existing = await knex('user_roles')
.where({ userId: data.userId, roleId })
.first();
if (existing) {
return { success: true, message: 'User already assigned' };
}
await knex('user_roles').insert({
id: knex.raw('(UUID())'),
userId: data.userId,
roleId,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
});
return { success: true };
}
@Delete(':roleId/users/:userId')
async removeUserFromRole(
@TenantId() tenantId: string,
@Param('roleId') roleId: string,
@Param('userId') userId: string,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
await knex('user_roles')
.where({ userId, roleId })
.delete();
return { success: true };
}
}

View File

@@ -1,148 +0,0 @@
import {
Controller,
Get,
Post,
Patch,
Delete,
Param,
Body,
UseGuards,
} from '@nestjs/common';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { TenantId } from '../tenant/tenant.decorator';
import { TenantDatabaseService } from '../tenant/tenant-database.service';
import { User } from '../models/user.model';
import * as bcrypt from 'bcrypt';
@Controller('setup/users')
@UseGuards(JwtAuthGuard)
export class SetupUsersController {
constructor(private tenantDbService: TenantDatabaseService) {}
@Get()
async getUsers(@TenantId() tenantId: string) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
return await User.query(knex).withGraphFetched('roles');
}
@Get(':id')
async getUser(
@TenantId() tenantId: string,
@Param('id') id: string,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
return await User.query(knex).findById(id).withGraphFetched('roles');
}
@Post()
async createUser(
@TenantId() tenantId: string,
@Body() data: { email: string; password: string; firstName?: string; lastName?: string; alias?: string },
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Hash password
const hashedPassword = await bcrypt.hash(data.password, 10);
const user = await User.query(knex).insert({
email: data.email,
password: hashedPassword,
firstName: data.firstName,
lastName: data.lastName,
alias: data.alias,
isActive: true,
});
return user;
}
@Patch(':id')
async updateUser(
@TenantId() tenantId: string,
@Param('id') id: string,
@Body() data: { email?: string; password?: string; firstName?: string; lastName?: string; alias?: string },
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const updateData: any = {};
if (data.email) updateData.email = data.email;
if (data.firstName !== undefined) updateData.firstName = data.firstName;
if (data.lastName !== undefined) updateData.lastName = data.lastName;
if (data.alias !== undefined) updateData.alias = data.alias;
// Hash password if provided
if (data.password) {
updateData.password = await bcrypt.hash(data.password, 10);
}
const user = await User.query(knex).patchAndFetchById(id, updateData);
return user;
}
@Delete(':id')
async deleteUser(
@TenantId() tenantId: string,
@Param('id') id: string,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Delete user role assignments first
await knex('user_roles').where({ userId: id }).delete();
// Delete the user
await User.query(knex).deleteById(id);
return { success: true };
}
@Post(':userId/roles')
async addRoleToUser(
@TenantId() tenantId: string,
@Param('userId') userId: string,
@Body() data: { roleId: string },
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Check if assignment already exists
const existing = await knex('user_roles')
.where({ userId, roleId: data.roleId })
.first();
if (existing) {
return { success: true, message: 'Role already assigned' };
}
await knex('user_roles').insert({
id: knex.raw('(UUID())'),
userId,
roleId: data.roleId,
created_at: knex.fn.now(),
updated_at: knex.fn.now(),
});
return { success: true };
}
@Delete(':userId/roles/:roleId')
async removeRoleFromUser(
@TenantId() tenantId: string,
@Param('userId') userId: string,
@Param('roleId') roleId: string,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
await knex('user_roles')
.where({ userId, roleId })
.delete();
return { success: true };
}
}

View File

@@ -0,0 +1,243 @@
import {
Controller,
Get,
Post,
Patch,
Delete,
Body,
Param,
Query,
UseGuards,
ForbiddenException,
NotFoundException,
} from '@nestjs/common';
import { IsString, IsArray, IsOptional, IsDateString } from 'class-validator';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { CurrentUser } from '../auth/current-user.decorator';
import { TenantId } from '../tenant/tenant.decorator';
import { TenantDatabaseService } from '../tenant/tenant-database.service';
export class CreateShareDto {
@IsString()
objectApiName: string;
@IsString()
recordId: string;
@IsString()
granteeUserId: string;
@IsArray()
@IsString({ each: true })
actions: string[]; // ["read"], ["read", "update"], etc.
@IsOptional()
@IsArray()
@IsString({ each: true })
fields?: string[]; // Optional field scoping
@IsOptional()
@IsDateString()
expiresAt?: string;
}
export class UpdateShareDto {
@IsOptional()
@IsArray()
@IsString({ each: true })
actions?: string[];
@IsOptional()
@IsArray()
@IsString({ each: true })
fields?: string[];
@IsOptional()
@IsDateString()
expiresAt?: string;
}
@Controller('rbac/shares')
@UseGuards(JwtAuthGuard)
export class ShareController {
constructor(private tenantDbService: TenantDatabaseService) {}
/**
* Create a new share
* Only the owner (or users with share permission) can share a record
*/
@Post()
async create(
@TenantId() tenantId: string,
@CurrentUser() currentUser: any,
@Body() createDto: CreateShareDto,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Get object definition by apiName
const objectDef = await knex('object_definitions')
.where({ apiName: createDto.objectApiName })
.first();
if (!objectDef) {
throw new NotFoundException('Object definition not found');
}
// Get the table name for the object
const tableName = this.getTableName(createDto.objectApiName);
// Verify the user owns the record
const record = await knex(tableName)
.where({ id: createDto.recordId })
.first();
if (!record) {
throw new NotFoundException('Record not found');
}
if (record.ownerId !== currentUser.userId) {
throw new ForbiddenException('Only the record owner can share it');
}
// Create the share
const shareId = require('crypto').randomUUID();
await knex('record_shares').insert({
id: shareId,
object_definition_id: objectDef.id,
record_id: createDto.recordId,
grantee_user_id: createDto.granteeUserId,
granted_by_user_id: currentUser.userId,
actions: JSON.stringify(createDto.actions),
fields: createDto.fields ? JSON.stringify(createDto.fields) : null,
expires_at: createDto.expiresAt,
created_at: knex.fn.now(),
});
const share = await knex('record_shares').where({ id: shareId }).first();
return {
...share,
actions: typeof share.actions === 'string' ? JSON.parse(share.actions) : share.actions,
fields: share.fields ? (typeof share.fields === 'string' ? JSON.parse(share.fields) : share.fields) : null,
};
}
private getTableName(objectApiName: string): string {
const snakeCase = objectApiName
.replace(/([A-Z])/g, '_$1')
.toLowerCase()
.replace(/^_/, '');
if (snakeCase.endsWith('y')) {
return snakeCase.slice(0, -1) + 'ies';
} else if (snakeCase.endsWith('s')) {
return snakeCase;
} else {
return snakeCase + 's';
}
}
/**
* List shares for a specific record
* Only owner or users with access can see shares
*/
@Get(':objectApiName/:recordId')
async listForRecord(
@TenantId() tenantId: string,
@CurrentUser() currentUser: any,
@Param('objectApiName') objectApiName: string,
@Param('recordId') recordId: string,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Get object definition
const objectDef = await knex('object_definitions')
.where({ apiName: objectApiName })
.first();
if (!objectDef) {
throw new NotFoundException('Object definition not found');
}
// Get shares for this record
const shares = await knex('record_shares')
.where({
object_definition_id: objectDef.id,
record_id: recordId,
})
.whereNull('revoked_at')
.select('*');
// Fetch user details for each share
const sharesWithUsers = await Promise.all(
shares.map(async (share: any) => {
const granteeUser = await knex('users')
.where({ id: share.grantee_user_id })
.select('id', 'email', 'firstName', 'lastName', 'name')
.first();
const grantedByUser = await knex('users')
.where({ id: share.granted_by_user_id })
.select('id', 'email', 'firstName', 'lastName', 'name')
.first();
return {
id: share.id,
recordId: share.record_id,
actions: typeof share.actions === 'string' ? JSON.parse(share.actions) : share.actions,
fields: share.fields ? (typeof share.fields === 'string' ? JSON.parse(share.fields) : share.fields) : null,
expiresAt: share.expires_at,
createdAt: share.created_at,
granteeUser: {
id: granteeUser.id,
email: granteeUser.email,
name: granteeUser.firstName && granteeUser.lastName
? `${granteeUser.firstName} ${granteeUser.lastName}`
: granteeUser.name || granteeUser.email,
},
grantedByUser: {
id: grantedByUser.id,
email: grantedByUser.email,
name: grantedByUser.firstName && grantedByUser.lastName
? `${grantedByUser.firstName} ${grantedByUser.lastName}`
: grantedByUser.name || grantedByUser.email,
},
};
})
);
return sharesWithUsers;
}
/**
* Revoke a share (soft delete)
*/
@Delete(':id')
async revoke(
@TenantId() tenantId: string,
@CurrentUser() currentUser: any,
@Param('id') id: string,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const share = await knex('record_shares').where({ id }).first();
if (!share) {
throw new NotFoundException('Share not found');
}
// Only the grantor can revoke
if (share.granted_by_user_id !== currentUser.userId) {
throw new ForbiddenException('Unauthorized');
}
await knex('record_shares')
.where({ id })
.update({ revoked_at: knex.fn.now() });
return { success: true };
}
}

View File

@@ -0,0 +1,41 @@
import { Controller, Get, UseGuards } from '@nestjs/common';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { TenantId } from '../tenant/tenant.decorator';
import { CurrentUser } from '../auth/current-user.decorator';
import { TenantDatabaseService } from '../tenant/tenant-database.service';
import { User } from '../models/user.model';
@Controller('rbac/users')
@UseGuards(JwtAuthGuard)
export class UserController {
constructor(private tenantDbService: TenantDatabaseService) {}
@Get()
async getUsers(
@TenantId() tenantId: string,
@CurrentUser() currentUser: any,
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
// Get all active users from tenant database (excluding current user)
let query = User.query(knex)
.select('id', 'email', 'firstName', 'lastName')
.where('isActive', true);
// Exclude current user if we have their ID
if (currentUser?.userId) {
query = query.whereNot('id', currentUser.userId);
}
const users = await query;
return users.map((user) => ({
id: user.id,
email: user.email,
name: user.firstName && user.lastName
? `${user.firstName} ${user.lastName}`
: user.email,
}));
}
}

View File

@@ -1,53 +0,0 @@
import { IsString, IsNotEmpty, IsArray, IsOptional } from 'class-validator';
export class CreateSavedViewDto {
@IsString()
@IsNotEmpty()
name: string;
@IsString()
@IsNotEmpty()
objectApiName: string;
@IsArray()
filters: Array<{
field: string;
operator: string;
value?: any;
values?: any[];
from?: string;
to?: string;
}>;
@IsOptional()
sort?: { field: string; direction: 'asc' | 'desc' } | null;
@IsOptional()
@IsString()
description?: string;
}
export class UpdateSavedViewDto {
@IsOptional()
@IsString()
@IsNotEmpty()
name?: string;
@IsOptional()
@IsArray()
filters?: Array<{
field: string;
operator: string;
value?: any;
values?: any[];
from?: string;
to?: string;
}>;
@IsOptional()
sort?: { field: string; direction: 'asc' | 'desc' } | null;
@IsOptional()
@IsString()
description?: string;
}

View File

@@ -1,92 +0,0 @@
import {
Controller,
Get,
Post,
Patch,
Delete,
Body,
Param,
UseGuards,
ForbiddenException,
NotFoundException,
} from '@nestjs/common';
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
import { CurrentUser } from '../auth/current-user.decorator';
import { TenantId } from '../tenant/tenant.decorator';
import { SavedListViewService } from './saved-list-view.service';
import { CreateSavedViewDto, UpdateSavedViewDto } from './dto/saved-list-view.dto';
import { CreateRecordShareDto } from '../rbac/dto/create-record-share.dto';
@Controller('saved-views')
@UseGuards(JwtAuthGuard)
export class SavedListViewController {
constructor(private readonly savedListViewService: SavedListViewService) {}
@Get(':objectApiName')
findByObject(
@TenantId() tenantId: string,
@CurrentUser() user: any,
@Param('objectApiName') objectApiName: string,
) {
return this.savedListViewService.findByObject(tenantId, user.userId, objectApiName);
}
@Post()
create(
@TenantId() tenantId: string,
@CurrentUser() user: any,
@Body() dto: CreateSavedViewDto,
) {
return this.savedListViewService.create(tenantId, user.userId, dto);
}
@Patch(':id')
update(
@TenantId() tenantId: string,
@CurrentUser() user: any,
@Param('id') id: string,
@Body() dto: UpdateSavedViewDto,
) {
return this.savedListViewService.update(tenantId, user.userId, id, dto);
}
@Delete(':id')
remove(
@TenantId() tenantId: string,
@CurrentUser() user: any,
@Param('id') id: string,
) {
return this.savedListViewService.remove(tenantId, user.userId, id);
}
// ── Sharing endpoints (reuse record_shares table) ────────────────────────
@Get(':id/shares')
getShares(
@TenantId() tenantId: string,
@CurrentUser() user: any,
@Param('id') id: string,
) {
return this.savedListViewService.getShares(tenantId, user.userId, id);
}
@Post(':id/shares')
createShare(
@TenantId() tenantId: string,
@CurrentUser() user: any,
@Param('id') id: string,
@Body() dto: CreateRecordShareDto,
) {
return this.savedListViewService.createShare(tenantId, user.userId, id, dto);
}
@Delete(':id/shares/:shareId')
removeShare(
@TenantId() tenantId: string,
@CurrentUser() user: any,
@Param('id') id: string,
@Param('shareId') shareId: string,
) {
return this.savedListViewService.removeShare(tenantId, user.userId, id, shareId);
}
}

View File

@@ -1,12 +0,0 @@
import { Module } from '@nestjs/common';
import { SavedListViewService } from './saved-list-view.service';
import { SavedListViewController } from './saved-list-view.controller';
import { TenantModule } from '../tenant/tenant.module';
@Module({
imports: [TenantModule],
controllers: [SavedListViewController],
providers: [SavedListViewService],
exports: [SavedListViewService],
})
export class SavedListViewModule {}

View File

@@ -1,264 +0,0 @@
import { Injectable, NotFoundException, ForbiddenException, BadRequestException } from '@nestjs/common';
import { TenantDatabaseService } from '../tenant/tenant-database.service';
import { CreateSavedViewDto, UpdateSavedViewDto } from './dto/saved-list-view.dto';
import { RecordShare } from '../models/record-share.model';
import { ObjectDefinition } from '../models/object-definition.model';
@Injectable()
export class SavedListViewService {
constructor(private readonly tenantDbService: TenantDatabaseService) {}
// ── Helpers ──────────────────────────────────────────────────────────────
/**
* Resolves the system object_definition ID for SavedListView.
* This is needed to create record_shares rows for saved views.
*/
private async getSavedViewObjectDefId(knex: any): Promise<string> {
const objectDef = await ObjectDefinition.query(knex)
.findOne({ apiName: 'SavedListView' });
if (!objectDef) {
throw new BadRequestException(
'SavedListView system object not found. Please run migrations.',
);
}
return objectDef.id;
}
// ── CRUD ─────────────────────────────────────────────────────────────────
/**
* Returns all saved views visible to the user for a given object:
* - Views owned by the user
* - Views shared with the user via record_shares
*/
async findByObject(tenantId: string, userId: string, objectApiName: string) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const objectDefId = await this.getSavedViewObjectDefId(knex);
// IDs of views shared with this user via record_shares
const sharedViewIds = await RecordShare.query(knex)
.where({ objectDefinitionId: objectDefId, granteeUserId: userId })
.whereNull('revokedAt')
.where(builder => {
builder.whereNull('expiresAt').orWhere('expiresAt', '>', new Date());
})
.select('recordId');
const sharedIds = sharedViewIds.map((s: any) => s.recordId);
const rows = await knex('saved_list_views')
.where({ object_api_name: objectApiName })
.andWhere(function () {
this.where({ user_id: userId });
if (sharedIds.length > 0) {
this.orWhereIn('id', sharedIds);
}
})
.orderBy('created_at', 'asc');
return rows.map((r: any) => this.deserialize(r, userId));
}
async create(tenantId: string, userId: string, dto: CreateSavedViewDto) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const id = require('crypto').randomUUID();
await knex('saved_list_views').insert({
id,
name: dto.name,
object_api_name: dto.objectApiName,
user_id: userId,
is_shared: false,
strategy: 'query',
filters: JSON.stringify(dto.filters || []),
sort: dto.sort ? JSON.stringify(dto.sort) : null,
description: dto.description || null,
});
const row = await knex('saved_list_views').where({ id }).first();
return this.deserialize(row, userId);
}
async update(tenantId: string, userId: string, id: string, dto: UpdateSavedViewDto) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const existing = await knex('saved_list_views').where({ id }).first();
if (!existing) throw new NotFoundException(`Saved view ${id} not found`);
if (existing.user_id !== userId) {
throw new ForbiddenException('You can only modify views you own');
}
const updates: Record<string, any> = { updated_at: knex.fn.now() };
if (dto.name !== undefined) updates.name = dto.name;
if (dto.filters !== undefined) updates.filters = JSON.stringify(dto.filters);
if (dto.sort !== undefined) updates.sort = dto.sort ? JSON.stringify(dto.sort) : null;
if (dto.description !== undefined) updates.description = dto.description;
await knex('saved_list_views').where({ id }).update(updates);
const row = await knex('saved_list_views').where({ id }).first();
return this.deserialize(row, userId);
}
async remove(tenantId: string, userId: string, id: string) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const existing = await knex('saved_list_views').where({ id }).first();
if (!existing) throw new NotFoundException(`Saved view ${id} not found`);
if (existing.user_id !== userId) {
throw new ForbiddenException('You can only delete views you own');
}
// Also clean up any record_shares for this view
const objectDefId = await this.getSavedViewObjectDefId(knex);
await RecordShare.query(knex)
.where({ objectDefinitionId: objectDefId, recordId: id })
.delete();
await knex('saved_list_views').where({ id }).delete();
return { deleted: true };
}
// ── Sharing via record_shares ────────────────────────────────────────────
async getShares(tenantId: string, userId: string, viewId: string) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const view = await knex('saved_list_views').where({ id: viewId }).first();
if (!view) throw new NotFoundException('Saved view not found');
if (view.user_id !== userId) {
throw new ForbiddenException('Only the view owner can manage sharing');
}
const objectDefId = await this.getSavedViewObjectDefId(knex);
const shares = await RecordShare.query(knex)
.where({ objectDefinitionId: objectDefId, recordId: viewId })
.whereNull('revokedAt')
.where(builder => {
builder.whereNull('expiresAt').orWhere('expiresAt', '>', new Date());
})
.withGraphFetched('[granteeUser]')
.orderBy('createdAt', 'desc');
return shares;
}
async createShare(
tenantId: string,
userId: string,
viewId: string,
dto: { granteeUserId: string; canRead: boolean; canEdit: boolean; canDelete: boolean; expiresAt?: string },
) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const view = await knex('saved_list_views').where({ id: viewId }).first();
if (!view) throw new NotFoundException('Saved view not found');
if (view.user_id !== userId) {
throw new ForbiddenException('Only the view owner can share it');
}
if (dto.granteeUserId === userId) {
throw new BadRequestException('Cannot share a view with yourself');
}
const objectDefId = await this.getSavedViewObjectDefId(knex);
// Upsert: if non-revoked share already exists for this grantee, update it
const existing = await RecordShare.query(knex)
.where({
objectDefinitionId: objectDefId,
recordId: viewId,
granteeUserId: dto.granteeUserId,
})
.whereNull('revokedAt')
.first();
if (existing) {
await RecordShare.query(knex)
.patchAndFetchById(existing.id, {
accessLevel: {
canRead: dto.canRead,
canEdit: dto.canEdit,
canDelete: dto.canDelete,
},
expiresAt: dto.expiresAt
? (knex.raw('?', [new Date(dto.expiresAt).toISOString().slice(0, 19).replace('T', ' ')]) as any)
: null,
} as any);
return RecordShare.query(knex)
.findById(existing.id)
.withGraphFetched('[granteeUser]');
}
const share = await RecordShare.query(knex).insertAndFetch({
objectDefinitionId: objectDefId,
recordId: viewId,
granteeUserId: dto.granteeUserId,
grantedByUserId: userId,
accessLevel: {
canRead: dto.canRead,
canEdit: dto.canEdit,
canDelete: dto.canDelete,
},
expiresAt: dto.expiresAt
? (knex.raw('?', [new Date(dto.expiresAt).toISOString().slice(0, 19).replace('T', ' ')]) as any)
: null,
} as any);
return RecordShare.query(knex)
.findById(share.id)
.withGraphFetched('[granteeUser]');
}
async removeShare(tenantId: string, userId: string, viewId: string, shareId: string) {
const resolvedTenantId = await this.tenantDbService.resolveTenantId(tenantId);
const knex = await this.tenantDbService.getTenantKnexById(resolvedTenantId);
const view = await knex('saved_list_views').where({ id: viewId }).first();
if (!view) throw new NotFoundException('Saved view not found');
if (view.user_id !== userId) {
throw new ForbiddenException('Only the view owner can manage sharing');
}
const share = await RecordShare.query(knex).findById(shareId);
if (!share) throw new NotFoundException('Share not found');
// Soft-revoke
await RecordShare.query(knex)
.findById(shareId)
.patch({ revokedAt: knex.fn.now() } as any);
return { revoked: true };
}
// ── Serialisation ────────────────────────────────────────────────────────
private deserialize(row: any, currentUserId: string) {
return {
id: row.id,
name: row.name,
objectApiName: row.object_api_name,
userId: row.user_id,
isOwner: row.user_id === currentUserId,
isShared: Boolean(row.is_shared),
strategy: row.strategy,
filters: typeof row.filters === 'string' ? JSON.parse(row.filters) : (row.filters ?? []),
sort: row.sort
? (typeof row.sort === 'string' ? JSON.parse(row.sort) : row.sort)
: null,
description: row.description,
createdAt: row.created_at,
updatedAt: row.updated_at,
};
}
}

View File

@@ -1,8 +0,0 @@
import { Module } from '@nestjs/common';
import { MeilisearchService } from './meilisearch.service';
@Module({
providers: [MeilisearchService],
exports: [MeilisearchService],
})
export class MeilisearchModule {}

View File

@@ -1,483 +0,0 @@
import { Injectable, Logger } from '@nestjs/common';
import * as http from 'http';
import * as https from 'https';
type MeiliConfig = {
host: string;
apiKey?: string;
indexPrefix: string;
};
type HybridSearchOptions = {
embedder: string;
semanticRatio?: number;
};
type OpenAiEmbedderConfig = {
embedderName: string;
apiKey: string;
model: string;
documentTemplate: string;
};
@Injectable()
export class MeilisearchService {
private readonly logger = new Logger(MeilisearchService.name);
private readonly embedderCache = new Map<string, string>();
private vectorStoreEnabled = false;
isEnabled(): boolean {
return Boolean(this.getConfig());
}
async searchRecord(
tenantId: string,
objectApiName: string,
query: string,
displayField?: string,
): Promise<{ id: string; hit: any } | null> {
const config = this.getConfig();
if (!config) return null;
const indexName = this.buildIndexName(config, tenantId, objectApiName);
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/search`;
console.log('querying Meilisearch index:', { indexName, query, displayField });
try {
const response = await this.requestJson('POST', url, {
q: query,
limit: 5,
}, this.buildHeaders(config));
if (!this.isSuccessStatus(response.status)) {
this.logger.warn(
`Meilisearch query failed for index ${indexName}: ${response.status}`,
);
return null;
}
const hits = Array.isArray(response.body?.hits) ? response.body.hits : [];
if (hits.length === 0) return null;
if (displayField) {
const loweredQuery = query.toLowerCase();
const exactMatch = hits.find((hit: any) => {
const value = hit?.[displayField];
return value && String(value).toLowerCase() === loweredQuery;
});
if (exactMatch?.id) {
return { id: exactMatch.id, hit: exactMatch };
}
}
const match = hits[0];
if (match?.id) {
return { id: match.id, hit: match };
}
} catch (error) {
this.logger.warn(`Meilisearch lookup failed: ${error.message}`);
}
return null;
}
async searchRecords(
tenantId: string,
objectApiName: string,
query: string,
options?: { limit?: number; offset?: number },
): Promise<{ hits: any[]; total: number }> {
const config = this.getConfig();
if (!config) return { hits: [], total: 0 };
const indexName = this.buildIndexName(config, tenantId, objectApiName);
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/search`;
const limit = Number.isFinite(Number(options?.limit)) ? Number(options?.limit) : 20;
const offset = Number.isFinite(Number(options?.offset)) ? Number(options?.offset) : 0;
try {
const response = await this.requestJson('POST', url, {
q: query,
limit,
offset,
}, this.buildHeaders(config));
console.log('Meilisearch response body:', response.body);
if (!this.isSuccessStatus(response.status)) {
this.logger.warn(
`Meilisearch query failed for index ${indexName}: ${response.status}`,
);
return { hits: [], total: 0 };
}
const hits = Array.isArray(response.body?.hits) ? response.body.hits : [];
const total =
response.body?.estimatedTotalHits ??
response.body?.nbHits ??
hits.length;
return { hits, total };
} catch (error) {
this.logger.warn(`Meilisearch query failed: ${error.message}`);
return { hits: [], total: 0 };
}
}
async upsertRecord(
tenantId: string,
objectApiName: string,
record: Record<string, any>,
fieldsToIndex: string[],
): Promise<void> {
const config = this.getConfig();
if (!config || !record?.id) return;
const indexName = this.buildIndexName(config, tenantId, objectApiName);
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/documents?primaryKey=id`;
const document = this.pickRecordFields(record, fieldsToIndex);
try {
const response = await this.requestJson('POST', url, [document], this.buildHeaders(config));
if (!this.isSuccessStatus(response.status)) {
this.logger.warn(
`Meilisearch upsert failed for index ${indexName}: ${response.status}`,
);
}
} catch (error) {
this.logger.warn(`Meilisearch upsert failed: ${error.message}`);
}
}
async deleteRecord(
tenantId: string,
objectApiName: string,
recordId: string,
): Promise<void> {
const config = this.getConfig();
if (!config || !recordId) return;
const indexName = this.buildIndexName(config, tenantId, objectApiName);
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/documents/${encodeURIComponent(recordId)}`;
try {
const response = await this.requestJson('DELETE', url, undefined, this.buildHeaders(config));
if (!this.isSuccessStatus(response.status)) {
this.logger.warn(
`Meilisearch delete failed for index ${indexName}: ${response.status}`,
);
}
} catch (error) {
this.logger.warn(`Meilisearch delete failed: ${error.message}`);
}
}
buildSemanticChunkIndexName(tenantId: string): string {
const config = this.getConfig();
const prefix = config?.indexPrefix || 'tenant_';
return `${prefix}${tenantId}_semantic_chunks`.toLowerCase();
}
async upsertDocuments(indexName: string, documents: Record<string, any>[]): Promise<void> {
const config = this.getConfig();
if (!config || !Array.isArray(documents) || documents.length === 0) return;
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/documents?primaryKey=id`;
try {
const response = await this.requestJson('POST', url, documents, this.buildHeaders(config));
if (!this.isSuccessStatus(response.status)) {
this.logger.warn(`Meilisearch document upsert failed for index ${indexName}: ${response.status}`);
return;
}
// Meilisearch indexes (and embeds) documents asynchronously. Wait for the task
// to complete so callers can immediately search and see the new documents.
const taskUid = response.body?.taskUid ?? response.body?.uid;
if (Number.isFinite(Number(taskUid))) {
const succeeded = await this.waitForTask(config, Number(taskUid), 30000);
if (!succeeded) {
this.logger.warn(`Meilisearch indexing task did not succeed within timeout: taskUid=${taskUid} index=${indexName}`);
}
}
} catch (error) {
this.logger.warn(`Meilisearch document upsert failed: ${error.message}`);
}
}
async searchIndex(
indexName: string,
query: string,
limit = 20,
hybrid?: HybridSearchOptions,
): Promise<{ hits: any[]; total: number }> {
const config = this.getConfig();
if (!config) return { hits: [], total: 0 };
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/search`;
try {
const response = await this.requestJson(
'POST',
url,
{
q: query,
limit,
showRankingScore: true,
...(hybrid ? { hybrid, showRankingScoreDetails: true } : {}),
},
this.buildHeaders(config),
);
if (!this.isSuccessStatus(response.status)) {
this.logger.warn(
`Meilisearch search failed for index ${indexName}: ${response.status}`,
);
this.logger.warn(
`Meilisearch search payload: ${JSON.stringify({ q: query, limit, hybrid })}`,
);
this.logger.warn(
`Meilisearch search error body: ${JSON.stringify(response.body)}`,
);
// If hybrid is invalid (embedder missing), retry once without hybrid
if (hybrid && response.body?.code === 'invalid_embedder') {
const fallback = await this.requestJson(
'POST',
url,
{ q: query, limit },
this.buildHeaders(config),
);
if (this.isSuccessStatus(fallback.status)) {
const hits = Array.isArray(fallback.body?.hits) ? fallback.body.hits : [];
const total =
fallback.body?.estimatedTotalHits ?? fallback.body?.nbHits ?? hits.length;
this.logger.warn(
`Meilisearch hybrid failed; fell back to lexical search for index ${indexName}.`,
);
return { hits, total };
}
}
return { hits: [], total: 0 };
}
const hits = Array.isArray(response.body?.hits) ? response.body.hits : [];
const total = response.body?.estimatedTotalHits ?? response.body?.nbHits ?? hits.length;
return { hits, total };
} catch (error) {
this.logger.warn(`Meilisearch search failed: ${error.message}`);
return { hits: [], total: 0 };
}
}
private getConfig(): MeiliConfig | null {
const host = process.env.MEILI_HOST || process.env.MEILISEARCH_HOST;
if (!host) return null;
const trimmedHost = host.replace(/\/+$/, '');
const apiKey = process.env.MEILI_API_KEY || process.env.MEILISEARCH_API_KEY;
const indexPrefix = process.env.MEILI_INDEX_PREFIX || 'tenant_';
return { host: trimmedHost, apiKey, indexPrefix };
}
private buildIndexName(config: MeiliConfig, tenantId: string, objectApiName: string): string {
return `${config.indexPrefix}${tenantId}_${objectApiName}`.toLowerCase();
}
private buildHeaders(config: MeiliConfig): Record<string, string> {
const headers: Record<string, string> = {
'Content-Type': 'application/json',
Accept: 'application/json',
};
if (config.apiKey) {
headers['X-Meili-API-Key'] = config.apiKey;
headers.Authorization = `Bearer ${config.apiKey}`;
}
return headers;
}
private pickRecordFields(record: Record<string, any>, fields: string[]): Record<string, any> {
const document: Record<string, any> = { id: record.id };
for (const field of fields) {
if (record[field] !== undefined) {
document[field] = record[field];
}
}
return document;
}
private isSuccessStatus(status: number): boolean {
return status >= 200 && status < 300;
}
private requestJson(
method: 'POST' | 'DELETE' | 'PATCH' | 'GET',
url: string,
payload: any,
headers: Record<string, string>,
): Promise<{ status: number; body: any }> {
return new Promise((resolve, reject) => {
const parsedUrl = new URL(url);
const client = parsedUrl.protocol === 'https:' ? https : http;
const request = client.request(
{
method,
hostname: parsedUrl.hostname,
port: parsedUrl.port,
path: `${parsedUrl.pathname}${parsedUrl.search}`,
headers,
},
(response) => {
let data = '';
response.on('data', (chunk) => {
data += chunk;
});
response.on('end', () => {
if (!data) {
resolve({ status: response.statusCode || 0, body: null });
return;
}
try {
const body = JSON.parse(data);
resolve({ status: response.statusCode || 0, body });
} catch (error) {
reject(error);
}
});
},
);
request.on('error', reject);
if (payload !== undefined && method !== 'GET') {
request.write(JSON.stringify(payload));
}
request.end();
});
}
private async enableVectorStore(): Promise<void> {
// Temporarily disabled to avoid the overhead of checking on every save.
// Re-enable by removing the early return below.
return;
if (this.vectorStoreEnabled) return; // eslint-disable-line no-unreachable
const meiliConfig = this.getConfig();
if (!meiliConfig) return;
const url = `${meiliConfig.host}/experimental-features`;
try {
const response = await this.requestJson(
'PATCH',
url,
{ vectorStore: true },
this.buildHeaders(meiliConfig),
);
if (this.isSuccessStatus(response.status)) {
this.vectorStoreEnabled = true;
this.logger.log('Meilisearch vector store experimental feature enabled');
} else {
this.logger.warn(
`Failed to enable Meilisearch vector store: ${response.status} ${JSON.stringify(response.body)}`,
);
}
} catch (error) {
this.logger.warn(`Failed to enable Meilisearch vector store: ${error.message}`);
}
}
async ensureOpenAiEmbedder(
indexName: string,
config: OpenAiEmbedderConfig,
): Promise<boolean> {
const meiliConfig = this.getConfig();
if (!meiliConfig || !config?.apiKey) return false;
await this.enableVectorStore();
const signature = JSON.stringify({
embedderName: config.embedderName,
model: config.model,
documentTemplate: config.documentTemplate,
apiKey: config.apiKey,
});
const cacheKey = `${indexName}:${config.embedderName}`;
if (this.embedderCache.get(cacheKey) === signature) {
return true;
}
const url = `${meiliConfig.host}/indexes/${encodeURIComponent(indexName)}/settings/embedders`;
try {
const response = await this.requestJson(
'PATCH',
url,
{
[config.embedderName]: {
source: 'openAi',
model: config.model,
apiKey: config.apiKey,
documentTemplate: config.documentTemplate,
},
},
this.buildHeaders(meiliConfig),
);
if (!this.isSuccessStatus(response.status)) {
this.logger.warn(
`Meilisearch embedder update failed for index ${indexName}: ${response.status}`,
);
this.logger.warn(
`Meilisearch embedder error body: ${JSON.stringify(response.body)}`,
);
return false;
}
const taskUid = response.body?.taskUid ?? response.body?.uid;
if (Number.isFinite(Number(taskUid))) {
const succeeded = await this.waitForTask(meiliConfig, Number(taskUid), 8000);
if (!succeeded) {
this.logger.warn(`Meilisearch embedder task did not succeed: ${taskUid}`);
return false;
}
}
const hasEmbedder = await this.hasEmbedder(meiliConfig, indexName, config.embedderName);
if (!hasEmbedder) {
this.logger.warn(`Meilisearch embedder missing after update: ${config.embedderName}`);
return false;
}
this.embedderCache.set(cacheKey, signature);
return true;
} catch (error) {
this.logger.warn(`Meilisearch embedder update failed: ${error.message}`);
return false;
}
}
private async waitForTask(
config: MeiliConfig,
taskUid: number,
timeoutMs = 8000,
): Promise<boolean> {
const url = `${config.host}/tasks/${taskUid}`;
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const response = await this.requestJson('GET', url, undefined, this.buildHeaders(config));
if (!this.isSuccessStatus(response.status)) {
return false;
}
const status = response.body?.status;
if (status === 'succeeded') return true;
if (status === 'failed' || status === 'canceled') {
this.logger.warn(`Meilisearch task ${taskUid} failed: ${JSON.stringify(response.body?.error)}`);
return false;
}
await new Promise((resolve) => setTimeout(resolve, 300));
}
return false;
}
private async hasEmbedder(
config: MeiliConfig,
indexName: string,
embedderName: string,
): Promise<boolean> {
const url = `${config.host}/indexes/${encodeURIComponent(indexName)}/settings/embedders`;
const response = await this.requestJson('GET', url, undefined, this.buildHeaders(config));
if (!this.isSuccessStatus(response.status)) {
return false;
}
const embedders = response.body || {};
return Boolean(embedders && embedders[embedderName]);
}
}

View File

@@ -1,14 +1,15 @@
import Knex from 'knex';
import type { Knex as KnexType } from 'knex';
import { Model } from 'objection';
import { CentralTenant, CentralDomain, CentralUser } from '../models/central.model';
let centralKnex: Knex.Knex | null = null;
let centralKnex: KnexType | null = null;
/**
* Get or create a Knex instance for the central database
* This is used for Objection models that work with central entities
*/
export function getCentralKnex(): Knex.Knex {
export function getCentralKnex(): KnexType {
if (!centralKnex) {
const centralDbUrl = process.env.CENTRAL_DATABASE_URL;

View File

@@ -242,26 +242,4 @@ export class TenantDatabaseService {
decrypted += decipher.final('utf8');
return decrypted;
}
/**
* Encrypt integrations config JSON object
* @param config - Plain object containing integration credentials
* @returns Encrypted JSON string
*/
encryptIntegrationsConfig(config: any): string {
if (!config) return null;
const jsonString = JSON.stringify(config);
return this.encryptPassword(jsonString);
}
/**
* Decrypt integrations config JSON string
* @param encryptedConfig - Encrypted JSON string
* @returns Plain object with integration credentials
*/
decryptIntegrationsConfig(encryptedConfig: string): any {
if (!encryptedConfig) return null;
const decrypted = this.decryptPassword(encryptedConfig);
return JSON.parse(decrypted);
}
}

View File

@@ -176,7 +176,7 @@ export class TenantProvisioningService {
* Seed default data for new tenant
*/
private async seedDefaultData(tenantId: string) {
const tenantKnex = await this.tenantDbService.getTenantKnexById(tenantId);
const tenantKnex = await this.tenantDbService.getTenantKnex(tenantId);
try {
// Create default roles

Some files were not shown because too many files have changed in this diff Show More