WIP - using deep agent to create dog using workflow
This commit is contained in:
Francisco Gaona
324
AI_PROCESS_BUILDER_README.md
Normal file
324
AI_PROCESS_BUILDER_README.md
Normal file
@@ -0,0 +1,324 @@
|
||||
# AI Process Builder + Chat Orchestrator
|
||||
|
||||
A complete implementation of tenant-scoped AI process automation where admins design LangGraph-compiled workflows via React Flow UI, and end-users execute them through a Deep Agent chat orchestrator with deterministic, audited execution.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### Backend Components
|
||||
|
||||
#### 1. **Deep Agent Orchestrator** ([deep-agent.orchestrator.ts](backend/src/ai-processes/deep-agent.orchestrator.ts))
|
||||
- Uses LangChain/OpenAI to intelligently select processes
|
||||
- Extracts structured inputs from natural language
|
||||
- Generates friendly confirmation messages
|
||||
- Three-step workflow: discover → select → extract → execute
|
||||
|
||||
#### 2. **Graph Compiler** ([ai-processes.compiler.ts](backend/src/ai-processes/ai-processes.compiler.ts))
|
||||
- Validates ReactFlow JSON graphs (Start/End nodes, reachability, cycles)
|
||||
- Compiles to LangGraph-compatible state machines
|
||||
- Validates tool allowlist and JSON schemas (Ajv)
|
||||
- Persists compiled artifact for versioned execution
|
||||
|
||||
#### 3. **Runtime Executor** ([ai-processes.runner.ts](backend/src/ai-processes/ai-processes.runner.ts))
|
||||
- Executes compiled graphs deterministically
|
||||
- Implements 4 node types: LLMDecisionNode, ToolNode, HumanInputNode, End
|
||||
- Handles conditional edges via jsonlogic
|
||||
- Emits real-time events for streaming updates
|
||||
|
||||
#### 4. **Tool Registry** ([tools/tool-registry.ts](backend/src/ai-processes/tools/tool-registry.ts))
|
||||
- Tenant-scoped tool allowlist (database-backed via AiToolConfig)
|
||||
- Demo tools wrapping ObjectService (findAccount, createAccount, etc.)
|
||||
- Context injection (tenantId, userId, knex) for secure execution
|
||||
|
||||
#### 5. **Orchestrator Service** ([ai-processes.orchestrator.service.ts](backend/src/ai-processes/ai-processes.orchestrator.service.ts))
|
||||
- Integrates Deep Agent for process selection
|
||||
- Falls back to standard AI assistant when no processes configured
|
||||
- Manages chat sessions and message history
|
||||
- Streams execution events via SSE
|
||||
|
||||
### Frontend Components
|
||||
|
||||
#### 1. **AIChatBar** ([components/AIChatBar.vue](frontend/components/AIChatBar.vue))
|
||||
- Updated to call `/ai-processes/chat/messages` endpoint
|
||||
- SSE event stream consumer for real-time updates
|
||||
- Displays process selection, node execution, tool calls
|
||||
- Handles NEED_INPUT events for human-in-the-loop
|
||||
|
||||
#### 2. **Process Management UI** ([pages/ai-processes/](frontend/pages/ai-processes/))
|
||||
- List view: displays all processes with versions
|
||||
- Editor view: React Flow integration via iframe + postMessage
|
||||
- Test runner for quick validation
|
||||
|
||||
#### 3. **React Flow Editor** ([ai-processes-editor/src/App.tsx](frontend/ai-processes-editor/src/App.tsx))
|
||||
- Node palette: Start, LLMDecisionNode, ToolNode, HumanInputNode, End
|
||||
- Visual graph designer with drag-drop
|
||||
- Auto-saves to parent window via postMessage
|
||||
- Loads existing graphs for editing
|
||||
|
||||
### Data Models (Objection.js)
|
||||
|
||||
```typescript
|
||||
AiProcess
|
||||
├── id, tenantId, name, description, latestVersion
|
||||
└── relations: versions[], runs[]
|
||||
|
||||
AiProcessVersion
|
||||
├── id, tenantId, processId, version
|
||||
├── graphJson (ReactFlow definition)
|
||||
└── compiledJson (LangGraph artifact)
|
||||
|
||||
AiProcessRun
|
||||
├── id, tenantId, processId, version, status
|
||||
├── inputJson, outputJson, errorJson, stateJson
|
||||
└── currentNodeId (for resume)
|
||||
|
||||
AiChatSession
|
||||
├── id, tenantId, userId
|
||||
└── relations: messages[]
|
||||
|
||||
AiChatMessage
|
||||
├── id, sessionId, role, content
|
||||
└── timestamps
|
||||
|
||||
AiAuditEvent
|
||||
├── id, tenantId, runId, eventType
|
||||
└── payloadJson (full event data)
|
||||
|
||||
AiToolConfig
|
||||
├── id, tenantId, toolName, enabled
|
||||
└── configJson (tool-specific settings)
|
||||
```
|
||||
|
||||
## Demo Process: Register New Pet
|
||||
|
||||
A complete workflow demonstrating conditional logic and tool orchestration:
|
||||
|
||||
1. **Extract Info** (LLMDecisionNode)
|
||||
- Parses user message for pet + owner details
|
||||
- Outputs structured JSON with validation
|
||||
|
||||
2. **Find/Create Account** (Conditional)
|
||||
- Searches for existing account by name/email
|
||||
- Creates new account if not found
|
||||
- Merges results into state
|
||||
|
||||
3. **Find/Create Contact** (Conditional)
|
||||
- Searches for existing contact under account
|
||||
- Creates new contact if not found
|
||||
|
||||
4. **Create Pet** (ToolNode)
|
||||
- Inserts pet record linked to contact
|
||||
- Returns pet ID
|
||||
|
||||
### Seed the Demo Process
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
npm run migrate:tenant -- <tenant-slug>
|
||||
npm run seed:demo-process -- <tenant-slug>
|
||||
```
|
||||
|
||||
### Test the Demo Process
|
||||
|
||||
1. Navigate to `/ai-processes` in your tenant subdomain
|
||||
2. Open "Register New Pet" process
|
||||
3. Click "Test Run" or use the chat bar:
|
||||
|
||||
```
|
||||
User: "Register a dog named Max, breed Golden Retriever, age 3,
|
||||
owned by John Smith, email john@example.com"
|
||||
|
||||
Agent: 🔄 Selected process: Register New Pet
|
||||
I'll register Max (Golden Retriever, 3 years old) for John Smith.
|
||||
|
||||
⚙️ Executing step: Extract Info
|
||||
✓ Extracted pet details
|
||||
|
||||
🔧 Using tool: findAccount
|
||||
ℹ️ Account not found, creating new account
|
||||
|
||||
🔧 Using tool: createAccount
|
||||
✓ Created account for John Smith
|
||||
|
||||
🔧 Using tool: findContact
|
||||
ℹ️ Contact not found, creating new contact
|
||||
|
||||
🔧 Using tool: createContact
|
||||
✓ Created contact: John Smith
|
||||
|
||||
🔧 Using tool: createPet
|
||||
✓ Created pet: Max (ID: pet_1234567890)
|
||||
|
||||
✅ Process completed successfully!
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Process Management (Admin)
|
||||
|
||||
```typescript
|
||||
GET /tenants/:tenantId/ai-processes
|
||||
POST /tenants/:tenantId/ai-processes
|
||||
GET /tenants/:tenantId/ai-processes/:id
|
||||
POST /tenants/:tenantId/ai-processes/:id/versions
|
||||
GET /tenants/:tenantId/ai-processes/:id/versions
|
||||
|
||||
POST /tenants/:tenantId/ai-processes/:id/runs
|
||||
POST /tenants/:tenantId/ai-processes/runs/:runId/resume
|
||||
```
|
||||
|
||||
### Chat Orchestrator (End User)
|
||||
|
||||
```typescript
|
||||
POST /tenants/:tenantId/ai-processes/chat/messages
|
||||
SSE /tenants/:tenantId/ai-processes/stream?sessionId=xxx
|
||||
```
|
||||
|
||||
## Event Stream Types
|
||||
|
||||
```typescript
|
||||
type StreamEvent =
|
||||
| { type: 'agent_started' }
|
||||
| { type: 'processes_listed', data: { count: number } }
|
||||
| { type: 'process_selected', processId: string, version: number }
|
||||
| { type: 'agent_message', data: { message: string } }
|
||||
| { type: 'node_started', nodeId: string }
|
||||
| { type: 'node_completed', nodeId: string }
|
||||
| { type: 'tool_called', toolName: string, nodeId: string }
|
||||
| { type: 'llm_decision', nodeId: string, data: any }
|
||||
| { type: 'need_input', data: { prompt: string, schema: JSONSchema } }
|
||||
| { type: 'final', data: { output: any } }
|
||||
| { type: 'error', data: { error: string } }
|
||||
```
|
||||
|
||||
## Security & Guardrails
|
||||
|
||||
### 1. **Tenancy Isolation**
|
||||
- All queries filtered by `tenantId` (enforced in Objection models)
|
||||
- Tool context includes tenant scope
|
||||
- Database-per-tenant architecture (inherited from platform)
|
||||
|
||||
### 2. **Tool Allowlist**
|
||||
- Two-level validation:
|
||||
- Tenant-level: `AiToolConfig` table (enabled tools per tenant)
|
||||
- Compile-time: validates toolName exists in registry
|
||||
- Runtime check before tool execution
|
||||
|
||||
### 3. **Schema Validation**
|
||||
- LLMDecisionNode output validated against JSON Schema (Ajv)
|
||||
- HumanInputNode input validated before resume
|
||||
- Graph structure validated at compile time
|
||||
|
||||
### 4. **Audit Trail**
|
||||
- Every node execution logged to `ai_audit_events`
|
||||
- Includes: tool calls, LLM decisions, state mutations, errors
|
||||
- Queryable for compliance dashboards
|
||||
|
||||
### 5. **Versioning**
|
||||
- Immutable process versions (create-only)
|
||||
- Runs reference specific version number
|
||||
- Graph definition + compiled artifact stored together
|
||||
|
||||
## Running the System
|
||||
|
||||
### 1. **Run Migrations**
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
npm run migrate:tenant -- tenant1
|
||||
```
|
||||
|
||||
### 2. **Seed Demo Data**
|
||||
|
||||
```bash
|
||||
npm run seed:demo-process -- tenant1
|
||||
```
|
||||
|
||||
### 3. **Start Backend**
|
||||
|
||||
```bash
|
||||
npm run start:dev
|
||||
```
|
||||
|
||||
### 4. **Build Editor (if needed)**
|
||||
|
||||
```bash
|
||||
cd frontend/ai-processes-editor
|
||||
npm install
|
||||
npm run build
|
||||
```
|
||||
|
||||
### 5. **Start Frontend**
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm run dev
|
||||
```
|
||||
|
||||
### 6. **Access UI**
|
||||
|
||||
- Admin UI: `http://tenant1.localhost:3001/ai-processes`
|
||||
- Chat UI: Available in bottom drawer on any page (⌘K to toggle)
|
||||
|
||||
## Extension Points
|
||||
|
||||
### Adding New Node Types
|
||||
|
||||
1. Define type in [ai-processes.types.ts](backend/src/ai-processes/ai-processes.types.ts)
|
||||
2. Add schema validation in [ai-processes.schemas.ts](backend/src/ai-processes/ai-processes.schemas.ts)
|
||||
3. Implement executor in [ai-processes.runner.ts](backend/src/ai-processes/ai-processes.runner.ts)
|
||||
4. Add UI component in React Flow editor
|
||||
|
||||
### Adding New Tools
|
||||
|
||||
1. Implement handler in [tools/demo-tools.ts](backend/src/ai-processes/tools/demo-tools.ts)
|
||||
2. Register in `demoTools` export
|
||||
3. Add to tenant allowlist via UI or seed script
|
||||
4. Document input/output schema
|
||||
|
||||
### Custom LLM Decision Logic
|
||||
|
||||
Override `llmDecision` callback in [ai-processes.service.ts](backend/src/ai-processes/ai-processes.service.ts):
|
||||
|
||||
```typescript
|
||||
llmDecision: async (node, state) => {
|
||||
const prompt = renderTemplate(node.data.promptTemplate, state);
|
||||
const response = await callOpenAI(prompt, node.data.model);
|
||||
return validateAgainstSchema(response, node.data.outputSchema);
|
||||
}
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Process not appearing in chat
|
||||
|
||||
- Check: `npm run seed:demo-process` completed successfully
|
||||
- Verify: Process exists in database (`select * from ai_processes`)
|
||||
- Check: Tools enabled (`select * from ai_tool_configs`)
|
||||
|
||||
### Graph validation errors
|
||||
|
||||
- Ensure exactly one Start node
|
||||
- Ensure at least one End node
|
||||
- Check all edges reference valid node IDs
|
||||
- Verify tool names match registered tools
|
||||
|
||||
### SSE stream not working
|
||||
|
||||
- Check CORS settings for subdomain routing
|
||||
- Verify `sessionId` returned from initial message
|
||||
- Check browser console for connection errors
|
||||
- Fallback: use polling endpoint (TODO: implement)
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Enhanced Input Extraction**: Use Deep Agent to extract required fields per process
|
||||
2. **Visual Schema Builder**: UI for JSON Schema creation (drag-drop fields)
|
||||
3. **Conditional Edge Builder**: Visual jsonlogic editor
|
||||
4. **Process Analytics**: Dashboard showing run success rates, avg duration
|
||||
5. **Human-in-Loop UI**: Dynamic form renderer for HumanInputNode
|
||||
6. **Process Marketplace**: Share processes across tenants (with permissions)
|
||||
7. **Python Microservice**: Optional Python runtime for native LangGraph support
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
Reference in New Issue
Block a user