WIP - call connecting to softphone
This commit is contained in:
Francisco Gaona
83
DEBUG_INCOMING_CALL.md
Normal file
83
DEBUG_INCOMING_CALL.md
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
# Debugging Incoming Call Issue
|
||||||
|
|
||||||
|
## Current Problem
|
||||||
|
- Hear "Connecting to your call" message (TwiML is executing)
|
||||||
|
- No ring on mobile after "Connecting" message
|
||||||
|
- Click Accept button does nothing
|
||||||
|
- Call never connects
|
||||||
|
|
||||||
|
## Root Cause Hypothesis
|
||||||
|
The Twilio Device SDK is likely **NOT receiving the incoming call event** from Twilio's Signaling Server. This could be because:
|
||||||
|
|
||||||
|
1. **Identity Mismatch**: The Device's identity (from JWT token) doesn't match the `<Client>ID</Client>` in TwiML
|
||||||
|
2. **Device Not Registered**: Device registration isn't completing before the call arrives
|
||||||
|
3. **Twilio Signaling Issue**: Device isn't connected to Twilio Signaling Server
|
||||||
|
|
||||||
|
## How to Debug
|
||||||
|
|
||||||
|
### Step 1: Check Device Identity in Console
|
||||||
|
When you open the softphone dialog, **open Browser DevTools Console (F12)**
|
||||||
|
|
||||||
|
You should see logs like:
|
||||||
|
```
|
||||||
|
Token received, creating Device...
|
||||||
|
Token identity: e6d45fa3-a108-4085-81e5-a8e05e85e6fb
|
||||||
|
Token grants: {voice: {...}}
|
||||||
|
Registering Twilio Device...
|
||||||
|
✓ Twilio Device registered - ready to receive calls
|
||||||
|
Device identity: e6d45fa3-a108-4085-81e5-a8e05e85e6fb
|
||||||
|
Device state: ready
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note the Device identity value** - e.g., "e6d45fa3-a108-4085-81e5-a8e05e85e6fb"
|
||||||
|
|
||||||
|
### Step 2: Check Backend Logs
|
||||||
|
When you make an inbound call, look for backend logs showing:
|
||||||
|
|
||||||
|
```
|
||||||
|
╔════════════════════════════════════════╗
|
||||||
|
║ === INBOUND CALL RECEIVED ===
|
||||||
|
╚════════════════════════════════════════╝
|
||||||
|
...
|
||||||
|
Client IDs to dial: e6d45fa3-a108-4085-81e5-a8e05e85e6fb
|
||||||
|
First Client ID format check: "e6d45fa3-a108-4085-81e5-a8e05e85e6fb" (length: 36)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Compare Identities
|
||||||
|
The Device identity from frontend console MUST MATCH the Client ID from backend logs.
|
||||||
|
|
||||||
|
**If they match**: The issue is with Twilio Signaling or Device SDK configuration
|
||||||
|
**If they don't match**: We found the bug - identity mismatch
|
||||||
|
|
||||||
|
### Step 4: Monitor Incoming Event
|
||||||
|
When you make the inbound call, keep watching the browser console for:
|
||||||
|
|
||||||
|
```
|
||||||
|
🔔 Twilio Device INCOMING event received: {...}
|
||||||
|
```
|
||||||
|
|
||||||
|
**If this appears**: The Device SDK IS receiving the call, so the Accept button issue is frontend
|
||||||
|
**If this doesn't appear**: The Device SDK is NOT receiving the call, so it's an identity/registration issue
|
||||||
|
|
||||||
|
## What Changed
|
||||||
|
- Frontend now relies on **Twilio Device SDK `incoming` event** (not Socket.IO) for showing incoming call
|
||||||
|
- Added comprehensive logging to Device initialization
|
||||||
|
- Added logging to Accept button handler
|
||||||
|
- Backend logs Device ID format for comparison
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
1. Make an inbound call
|
||||||
|
2. Check browser console for the 5 logs above
|
||||||
|
3. Check backend logs for Client ID
|
||||||
|
4. Look for "🔔 Twilio Device INCOMING event" in browser console
|
||||||
|
5. Try clicking Accept and watch console for "📞 Accepting call" logs
|
||||||
|
6. Report back with:
|
||||||
|
- Device identity from console
|
||||||
|
- Client ID from backend logs
|
||||||
|
- Whether "🔔 Twilio Device INCOMING event" appears
|
||||||
|
- Whether any accept logs appear
|
||||||
|
|
||||||
|
## Important Files
|
||||||
|
- Backend: `/backend/src/voice/voice.controller.ts` (lines 205-210 show Client ID logging)
|
||||||
|
- Frontend: `/frontend/composables/useSoftphone.ts` (Device initialization and incoming handler)
|
||||||
249
backend/package-lock.json
generated
249
backend/package-lock.json
generated
@@ -10,7 +10,7 @@
|
|||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@casl/ability": "^6.7.5",
|
"@casl/ability": "^6.7.5",
|
||||||
"@fastify/websocket": "^11.2.0",
|
"@fastify/websocket": "^10.0.1",
|
||||||
"@nestjs/bullmq": "^10.1.0",
|
"@nestjs/bullmq": "^10.1.0",
|
||||||
"@nestjs/common": "^10.3.0",
|
"@nestjs/common": "^10.3.0",
|
||||||
"@nestjs/config": "^3.1.1",
|
"@nestjs/config": "^3.1.1",
|
||||||
@@ -19,6 +19,7 @@
|
|||||||
"@nestjs/passport": "^10.0.3",
|
"@nestjs/passport": "^10.0.3",
|
||||||
"@nestjs/platform-fastify": "^10.3.0",
|
"@nestjs/platform-fastify": "^10.3.0",
|
||||||
"@nestjs/platform-socket.io": "^10.4.20",
|
"@nestjs/platform-socket.io": "^10.4.20",
|
||||||
|
"@nestjs/serve-static": "^4.0.2",
|
||||||
"@nestjs/websockets": "^10.4.20",
|
"@nestjs/websockets": "^10.4.20",
|
||||||
"@prisma/client": "^5.8.0",
|
"@prisma/client": "^5.8.0",
|
||||||
"bcrypt": "^5.1.1",
|
"bcrypt": "^5.1.1",
|
||||||
@@ -981,42 +982,16 @@
|
|||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
"node_modules/@fastify/websocket": {
|
"node_modules/@fastify/websocket": {
|
||||||
"version": "11.2.0",
|
"version": "10.0.1",
|
||||||
"resolved": "https://registry.npmjs.org/@fastify/websocket/-/websocket-11.2.0.tgz",
|
"resolved": "https://registry.npmjs.org/@fastify/websocket/-/websocket-10.0.1.tgz",
|
||||||
"integrity": "sha512-3HrDPbAG1CzUCqnslgJxppvzaAZffieOVbLp1DAy1huCSynUWPifSvfdEDUR8HlJLp3sp1A36uOM2tJogADS8w==",
|
"integrity": "sha512-8/pQIxTPRD8U94aILTeJ+2O3el/r19+Ej5z1O1mXlqplsUH7KzCjAI0sgd5DM/NoPjAi5qLFNIjgM5+9/rGSNw==",
|
||||||
"funding": [
|
|
||||||
{
|
|
||||||
"type": "github",
|
|
||||||
"url": "https://github.com/sponsors/fastify"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"type": "opencollective",
|
|
||||||
"url": "https://opencollective.com/fastify"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"duplexify": "^4.1.3",
|
"duplexify": "^4.1.2",
|
||||||
"fastify-plugin": "^5.0.0",
|
"fastify-plugin": "^4.0.0",
|
||||||
"ws": "^8.16.0"
|
"ws": "^8.0.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/@fastify/websocket/node_modules/fastify-plugin": {
|
|
||||||
"version": "5.1.0",
|
|
||||||
"resolved": "https://registry.npmjs.org/fastify-plugin/-/fastify-plugin-5.1.0.tgz",
|
|
||||||
"integrity": "sha512-FAIDA8eovSt5qcDgcBvDuX/v0Cjz0ohGhENZ/wpc3y+oZCY2afZ9Baqql3g/lC+OHRnciQol4ww7tuthOb9idw==",
|
|
||||||
"funding": [
|
|
||||||
{
|
|
||||||
"type": "github",
|
|
||||||
"url": "https://github.com/sponsors/fastify"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"type": "opencollective",
|
|
||||||
"url": "https://opencollective.com/fastify"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"license": "MIT"
|
|
||||||
},
|
|
||||||
"node_modules/@humanwhocodes/config-array": {
|
"node_modules/@humanwhocodes/config-array": {
|
||||||
"version": "0.13.0",
|
"version": "0.13.0",
|
||||||
"resolved": "https://registry.npmjs.org/@humanwhocodes/config-array/-/config-array-0.13.0.tgz",
|
"resolved": "https://registry.npmjs.org/@humanwhocodes/config-array/-/config-array-0.13.0.tgz",
|
||||||
@@ -2200,6 +2175,39 @@
|
|||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "MIT"
|
"license": "MIT"
|
||||||
},
|
},
|
||||||
|
"node_modules/@nestjs/serve-static": {
|
||||||
|
"version": "4.0.2",
|
||||||
|
"resolved": "https://registry.npmjs.org/@nestjs/serve-static/-/serve-static-4.0.2.tgz",
|
||||||
|
"integrity": "sha512-cT0vdWN5ar7jDI2NKbhf4LcwJzU4vS5sVpMkVrHuyLcltbrz6JdGi1TfIMMatP2pNiq5Ie/uUdPSFDVaZX/URQ==",
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"path-to-regexp": "0.2.5"
|
||||||
|
},
|
||||||
|
"peerDependencies": {
|
||||||
|
"@fastify/static": "^6.5.0 || ^7.0.0",
|
||||||
|
"@nestjs/common": "^9.0.0 || ^10.0.0",
|
||||||
|
"@nestjs/core": "^9.0.0 || ^10.0.0",
|
||||||
|
"express": "^4.18.1",
|
||||||
|
"fastify": "^4.7.0"
|
||||||
|
},
|
||||||
|
"peerDependenciesMeta": {
|
||||||
|
"@fastify/static": {
|
||||||
|
"optional": true
|
||||||
|
},
|
||||||
|
"express": {
|
||||||
|
"optional": true
|
||||||
|
},
|
||||||
|
"fastify": {
|
||||||
|
"optional": true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/@nestjs/serve-static/node_modules/path-to-regexp": {
|
||||||
|
"version": "0.2.5",
|
||||||
|
"resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.2.5.tgz",
|
||||||
|
"integrity": "sha512-l6qtdDPIkmAmzEO6egquYDfqQGPMRNGjYtrU13HAXb3YSRrt7HSb1sJY0pKp6o2bAa86tSB6iwaW2JbthPKr7Q==",
|
||||||
|
"license": "MIT"
|
||||||
|
},
|
||||||
"node_modules/@nestjs/testing": {
|
"node_modules/@nestjs/testing": {
|
||||||
"version": "10.4.20",
|
"version": "10.4.20",
|
||||||
"resolved": "https://registry.npmjs.org/@nestjs/testing/-/testing-10.4.20.tgz",
|
"resolved": "https://registry.npmjs.org/@nestjs/testing/-/testing-10.4.20.tgz",
|
||||||
@@ -2359,14 +2367,14 @@
|
|||||||
"version": "5.22.0",
|
"version": "5.22.0",
|
||||||
"resolved": "https://registry.npmjs.org/@prisma/debug/-/debug-5.22.0.tgz",
|
"resolved": "https://registry.npmjs.org/@prisma/debug/-/debug-5.22.0.tgz",
|
||||||
"integrity": "sha512-AUt44v3YJeggO2ZU5BkXI7M4hu9BF2zzH2iF2V5pyXT/lRTyWiElZ7It+bRH1EshoMRxHgpYg4VB6rCM+mG5jQ==",
|
"integrity": "sha512-AUt44v3YJeggO2ZU5BkXI7M4hu9BF2zzH2iF2V5pyXT/lRTyWiElZ7It+bRH1EshoMRxHgpYg4VB6rCM+mG5jQ==",
|
||||||
"dev": true,
|
"devOptional": true,
|
||||||
"license": "Apache-2.0"
|
"license": "Apache-2.0"
|
||||||
},
|
},
|
||||||
"node_modules/@prisma/engines": {
|
"node_modules/@prisma/engines": {
|
||||||
"version": "5.22.0",
|
"version": "5.22.0",
|
||||||
"resolved": "https://registry.npmjs.org/@prisma/engines/-/engines-5.22.0.tgz",
|
"resolved": "https://registry.npmjs.org/@prisma/engines/-/engines-5.22.0.tgz",
|
||||||
"integrity": "sha512-UNjfslWhAt06kVL3CjkuYpHAWSO6L4kDCVPegV6itt7nD1kSJavd3vhgAEhjglLJJKEdJ7oIqDJ+yHk6qO8gPA==",
|
"integrity": "sha512-UNjfslWhAt06kVL3CjkuYpHAWSO6L4kDCVPegV6itt7nD1kSJavd3vhgAEhjglLJJKEdJ7oIqDJ+yHk6qO8gPA==",
|
||||||
"dev": true,
|
"devOptional": true,
|
||||||
"hasInstallScript": true,
|
"hasInstallScript": true,
|
||||||
"license": "Apache-2.0",
|
"license": "Apache-2.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
@@ -2380,14 +2388,14 @@
|
|||||||
"version": "5.22.0-44.605197351a3c8bdd595af2d2a9bc3025bca48ea2",
|
"version": "5.22.0-44.605197351a3c8bdd595af2d2a9bc3025bca48ea2",
|
||||||
"resolved": "https://registry.npmjs.org/@prisma/engines-version/-/engines-version-5.22.0-44.605197351a3c8bdd595af2d2a9bc3025bca48ea2.tgz",
|
"resolved": "https://registry.npmjs.org/@prisma/engines-version/-/engines-version-5.22.0-44.605197351a3c8bdd595af2d2a9bc3025bca48ea2.tgz",
|
||||||
"integrity": "sha512-2PTmxFR2yHW/eB3uqWtcgRcgAbG1rwG9ZriSvQw+nnb7c4uCr3RAcGMb6/zfE88SKlC1Nj2ziUvc96Z379mHgQ==",
|
"integrity": "sha512-2PTmxFR2yHW/eB3uqWtcgRcgAbG1rwG9ZriSvQw+nnb7c4uCr3RAcGMb6/zfE88SKlC1Nj2ziUvc96Z379mHgQ==",
|
||||||
"dev": true,
|
"devOptional": true,
|
||||||
"license": "Apache-2.0"
|
"license": "Apache-2.0"
|
||||||
},
|
},
|
||||||
"node_modules/@prisma/fetch-engine": {
|
"node_modules/@prisma/fetch-engine": {
|
||||||
"version": "5.22.0",
|
"version": "5.22.0",
|
||||||
"resolved": "https://registry.npmjs.org/@prisma/fetch-engine/-/fetch-engine-5.22.0.tgz",
|
"resolved": "https://registry.npmjs.org/@prisma/fetch-engine/-/fetch-engine-5.22.0.tgz",
|
||||||
"integrity": "sha512-bkrD/Mc2fSvkQBV5EpoFcZ87AvOgDxbG99488a5cexp5Ccny+UM6MAe/UFkUC0wLYD9+9befNOqGiIJhhq+HbA==",
|
"integrity": "sha512-bkrD/Mc2fSvkQBV5EpoFcZ87AvOgDxbG99488a5cexp5Ccny+UM6MAe/UFkUC0wLYD9+9befNOqGiIJhhq+HbA==",
|
||||||
"dev": true,
|
"devOptional": true,
|
||||||
"license": "Apache-2.0",
|
"license": "Apache-2.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@prisma/debug": "5.22.0",
|
"@prisma/debug": "5.22.0",
|
||||||
@@ -2399,7 +2407,7 @@
|
|||||||
"version": "5.22.0",
|
"version": "5.22.0",
|
||||||
"resolved": "https://registry.npmjs.org/@prisma/get-platform/-/get-platform-5.22.0.tgz",
|
"resolved": "https://registry.npmjs.org/@prisma/get-platform/-/get-platform-5.22.0.tgz",
|
||||||
"integrity": "sha512-pHhpQdr1UPFpt+zFfnPazhulaZYCUqeIcPpJViYoq9R+D/yw4fjE+CtnsnKzPYm0ddUbeXUzjGVGIRVgPDCk4Q==",
|
"integrity": "sha512-pHhpQdr1UPFpt+zFfnPazhulaZYCUqeIcPpJViYoq9R+D/yw4fjE+CtnsnKzPYm0ddUbeXUzjGVGIRVgPDCk4Q==",
|
||||||
"dev": true,
|
"devOptional": true,
|
||||||
"license": "Apache-2.0",
|
"license": "Apache-2.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@prisma/debug": "5.22.0"
|
"@prisma/debug": "5.22.0"
|
||||||
@@ -3286,6 +3294,20 @@
|
|||||||
"node": ">=0.4.0"
|
"node": ">=0.4.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/acorn-import-phases": {
|
||||||
|
"version": "1.0.4",
|
||||||
|
"resolved": "https://registry.npmjs.org/acorn-import-phases/-/acorn-import-phases-1.0.4.tgz",
|
||||||
|
"integrity": "sha512-wKmbr/DDiIXzEOiWrTTUcDm24kQ2vGfZQvM2fwg2vXqR5uW6aapr7ObPtj1th32b9u90/Pf4AItvdTh42fBmVQ==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"peer": true,
|
||||||
|
"engines": {
|
||||||
|
"node": ">=10.13.0"
|
||||||
|
},
|
||||||
|
"peerDependencies": {
|
||||||
|
"acorn": "^8.14.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/acorn-jsx": {
|
"node_modules/acorn-jsx": {
|
||||||
"version": "5.3.2",
|
"version": "5.3.2",
|
||||||
"resolved": "https://registry.npmjs.org/acorn-jsx/-/acorn-jsx-5.3.2.tgz",
|
"resolved": "https://registry.npmjs.org/acorn-jsx/-/acorn-jsx-5.3.2.tgz",
|
||||||
@@ -3716,9 +3738,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/baseline-browser-mapping": {
|
"node_modules/baseline-browser-mapping": {
|
||||||
"version": "2.8.31",
|
"version": "2.9.11",
|
||||||
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.8.31.tgz",
|
"resolved": "https://registry.npmjs.org/baseline-browser-mapping/-/baseline-browser-mapping-2.9.11.tgz",
|
||||||
"integrity": "sha512-a28v2eWrrRWPpJSzxc+mKwm0ZtVx/G8SepdQZDArnXYU/XS+IF6mp8aB/4E+hH1tyGCoDo3KlUCdlSxGDsRkAw==",
|
"integrity": "sha512-Sg0xJUNDU1sJNGdfGWhVHX0kkZ+HWcvmVymJbj6NSgZZmW/8S9Y2HQ5euytnIgakgxN6papOAWiwDo1ctFDcoQ==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "Apache-2.0",
|
"license": "Apache-2.0",
|
||||||
"bin": {
|
"bin": {
|
||||||
@@ -3788,9 +3810,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/browserslist": {
|
"node_modules/browserslist": {
|
||||||
"version": "4.28.0",
|
"version": "4.28.1",
|
||||||
"resolved": "https://registry.npmjs.org/browserslist/-/browserslist-4.28.0.tgz",
|
"resolved": "https://registry.npmjs.org/browserslist/-/browserslist-4.28.1.tgz",
|
||||||
"integrity": "sha512-tbydkR/CxfMwelN0vwdP/pLkDwyAASZ+VfWm4EOwlB6SWhx1sYnWLqo8N5j0rAzPfzfRaxt0mM/4wPU/Su84RQ==",
|
"integrity": "sha512-ZC5Bd0LgJXgwGqUknZY/vkUQ04r8NXnJZ3yYi4vDmSiZmC/pdSN0NbNRPxZpbtO4uAfDUAFffO8IZoM3Gj8IkA==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"funding": [
|
"funding": [
|
||||||
{
|
{
|
||||||
@@ -3808,11 +3830,11 @@
|
|||||||
],
|
],
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"baseline-browser-mapping": "^2.8.25",
|
"baseline-browser-mapping": "^2.9.0",
|
||||||
"caniuse-lite": "^1.0.30001754",
|
"caniuse-lite": "^1.0.30001759",
|
||||||
"electron-to-chromium": "^1.5.249",
|
"electron-to-chromium": "^1.5.263",
|
||||||
"node-releases": "^2.0.27",
|
"node-releases": "^2.0.27",
|
||||||
"update-browserslist-db": "^1.1.4"
|
"update-browserslist-db": "^1.2.0"
|
||||||
},
|
},
|
||||||
"bin": {
|
"bin": {
|
||||||
"browserslist": "cli.js"
|
"browserslist": "cli.js"
|
||||||
@@ -3966,9 +3988,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/caniuse-lite": {
|
"node_modules/caniuse-lite": {
|
||||||
"version": "1.0.30001757",
|
"version": "1.0.30001762",
|
||||||
"resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001757.tgz",
|
"resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001762.tgz",
|
||||||
"integrity": "sha512-r0nnL/I28Zi/yjk1el6ilj27tKcdjLsNqAOZr0yVjWPrSQyHgKI2INaEWw21bAQSv2LXRt1XuCS/GomNpWOxsQ==",
|
"integrity": "sha512-PxZwGNvH7Ak8WX5iXzoK1KPZttBXNPuaOvI2ZYU7NrlM+d9Ov+TUvlLOBNGzVXAntMSMMlJPd+jY6ovrVjSmUw==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"funding": [
|
"funding": [
|
||||||
{
|
{
|
||||||
@@ -4679,9 +4701,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/electron-to-chromium": {
|
"node_modules/electron-to-chromium": {
|
||||||
"version": "1.5.260",
|
"version": "1.5.267",
|
||||||
"resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.5.260.tgz",
|
"resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.5.267.tgz",
|
||||||
"integrity": "sha512-ov8rBoOBhVawpzdre+Cmz4FB+y66Eqrk6Gwqd8NGxuhv99GQ8XqMAr351KEkOt7gukXWDg6gJWEMKgL2RLMPtA==",
|
"integrity": "sha512-0Drusm6MVRXSOJpGbaSVgcQsuB4hEkMpHXaVstcPmhu5LIedxs1xNK/nIxmQIU/RPC0+1/o0AVZfBTkTNJOdUw==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "ISC"
|
"license": "ISC"
|
||||||
},
|
},
|
||||||
@@ -8852,7 +8874,7 @@
|
|||||||
"version": "5.22.0",
|
"version": "5.22.0",
|
||||||
"resolved": "https://registry.npmjs.org/prisma/-/prisma-5.22.0.tgz",
|
"resolved": "https://registry.npmjs.org/prisma/-/prisma-5.22.0.tgz",
|
||||||
"integrity": "sha512-vtpjW3XuYCSnMsNVBjLMNkTj6OZbudcPPTPYHqX0CJfpcdWciI1dM8uHETwmDxxiqEwCIE6WvXucWUetJgfu/A==",
|
"integrity": "sha512-vtpjW3XuYCSnMsNVBjLMNkTj6OZbudcPPTPYHqX0CJfpcdWciI1dM8uHETwmDxxiqEwCIE6WvXucWUetJgfu/A==",
|
||||||
"dev": true,
|
"devOptional": true,
|
||||||
"hasInstallScript": true,
|
"hasInstallScript": true,
|
||||||
"license": "Apache-2.0",
|
"license": "Apache-2.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
@@ -10006,9 +10028,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/terser-webpack-plugin": {
|
"node_modules/terser-webpack-plugin": {
|
||||||
"version": "5.3.14",
|
"version": "5.3.16",
|
||||||
"resolved": "https://registry.npmjs.org/terser-webpack-plugin/-/terser-webpack-plugin-5.3.14.tgz",
|
"resolved": "https://registry.npmjs.org/terser-webpack-plugin/-/terser-webpack-plugin-5.3.16.tgz",
|
||||||
"integrity": "sha512-vkZjpUjb6OMS7dhV+tILUW6BhpDR7P2L/aQSAv+Uwk+m8KATX9EccViHTJR2qDtACKPIYndLGCyl3FMo+r2LMw==",
|
"integrity": "sha512-h9oBFCWrq78NyWWVcSwZarJkZ01c2AyGrzs1crmHZO3QUg9D61Wu4NPjBy69n7JqylFF5y+CsUZYmYEIZ3mR+Q==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
@@ -10581,9 +10603,9 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/update-browserslist-db": {
|
"node_modules/update-browserslist-db": {
|
||||||
"version": "1.1.4",
|
"version": "1.2.3",
|
||||||
"resolved": "https://registry.npmjs.org/update-browserslist-db/-/update-browserslist-db-1.1.4.tgz",
|
"resolved": "https://registry.npmjs.org/update-browserslist-db/-/update-browserslist-db-1.2.3.tgz",
|
||||||
"integrity": "sha512-q0SPT4xyU84saUX+tomz1WLkxUbuaJnR1xWt17M7fJtEJigJeWUNGUqrauFXsHnqev9y9JTRGwk13tFBuKby4A==",
|
"integrity": "sha512-Js0m9cx+qOgDxo0eMiFGEueWztz+d4+M3rGlmKPT+T4IS/jP4ylw3Nwpu6cpTTP8R1MAC1kF4VbdLt3ARf209w==",
|
||||||
"dev": true,
|
"dev": true,
|
||||||
"funding": [
|
"funding": [
|
||||||
{
|
{
|
||||||
@@ -10728,6 +10750,56 @@
|
|||||||
"integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==",
|
"integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==",
|
||||||
"license": "BSD-2-Clause"
|
"license": "BSD-2-Clause"
|
||||||
},
|
},
|
||||||
|
"node_modules/webpack": {
|
||||||
|
"version": "5.104.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/webpack/-/webpack-5.104.1.tgz",
|
||||||
|
"integrity": "sha512-Qphch25abbMNtekmEGJmeRUhLDbe+QfiWTiqpKYkpCOWY64v9eyl+KRRLmqOFA2AvKPpc9DC6+u2n76tQLBoaA==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"peer": true,
|
||||||
|
"dependencies": {
|
||||||
|
"@types/eslint-scope": "^3.7.7",
|
||||||
|
"@types/estree": "^1.0.8",
|
||||||
|
"@types/json-schema": "^7.0.15",
|
||||||
|
"@webassemblyjs/ast": "^1.14.1",
|
||||||
|
"@webassemblyjs/wasm-edit": "^1.14.1",
|
||||||
|
"@webassemblyjs/wasm-parser": "^1.14.1",
|
||||||
|
"acorn": "^8.15.0",
|
||||||
|
"acorn-import-phases": "^1.0.3",
|
||||||
|
"browserslist": "^4.28.1",
|
||||||
|
"chrome-trace-event": "^1.0.2",
|
||||||
|
"enhanced-resolve": "^5.17.4",
|
||||||
|
"es-module-lexer": "^2.0.0",
|
||||||
|
"eslint-scope": "5.1.1",
|
||||||
|
"events": "^3.2.0",
|
||||||
|
"glob-to-regexp": "^0.4.1",
|
||||||
|
"graceful-fs": "^4.2.11",
|
||||||
|
"json-parse-even-better-errors": "^2.3.1",
|
||||||
|
"loader-runner": "^4.3.1",
|
||||||
|
"mime-types": "^2.1.27",
|
||||||
|
"neo-async": "^2.6.2",
|
||||||
|
"schema-utils": "^4.3.3",
|
||||||
|
"tapable": "^2.3.0",
|
||||||
|
"terser-webpack-plugin": "^5.3.16",
|
||||||
|
"watchpack": "^2.4.4",
|
||||||
|
"webpack-sources": "^3.3.3"
|
||||||
|
},
|
||||||
|
"bin": {
|
||||||
|
"webpack": "bin/webpack.js"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=10.13.0"
|
||||||
|
},
|
||||||
|
"funding": {
|
||||||
|
"type": "opencollective",
|
||||||
|
"url": "https://opencollective.com/webpack"
|
||||||
|
},
|
||||||
|
"peerDependenciesMeta": {
|
||||||
|
"webpack-cli": {
|
||||||
|
"optional": true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/webpack-node-externals": {
|
"node_modules/webpack-node-externals": {
|
||||||
"version": "3.0.0",
|
"version": "3.0.0",
|
||||||
"resolved": "https://registry.npmjs.org/webpack-node-externals/-/webpack-node-externals-3.0.0.tgz",
|
"resolved": "https://registry.npmjs.org/webpack-node-externals/-/webpack-node-externals-3.0.0.tgz",
|
||||||
@@ -10748,6 +10820,61 @@
|
|||||||
"node": ">=10.13.0"
|
"node": ">=10.13.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/webpack/node_modules/es-module-lexer": {
|
||||||
|
"version": "2.0.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.0.0.tgz",
|
||||||
|
"integrity": "sha512-5POEcUuZybH7IdmGsD8wlf0AI55wMecM9rVBTI/qEAy2c1kTOm3DjFYjrBdI2K3BaJjJYfYFeRtM0t9ssnRuxw==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"peer": true
|
||||||
|
},
|
||||||
|
"node_modules/webpack/node_modules/eslint-scope": {
|
||||||
|
"version": "5.1.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/eslint-scope/-/eslint-scope-5.1.1.tgz",
|
||||||
|
"integrity": "sha512-2NxwbF/hZ0KpepYN0cNbo+FN6XoK7GaHlQhgx/hIZl6Va0bF45RQOOwhLIy8lQDbuCiadSLCBnH2CFYquit5bw==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "BSD-2-Clause",
|
||||||
|
"peer": true,
|
||||||
|
"dependencies": {
|
||||||
|
"esrecurse": "^4.3.0",
|
||||||
|
"estraverse": "^4.1.1"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=8.0.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/webpack/node_modules/estraverse": {
|
||||||
|
"version": "4.3.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/estraverse/-/estraverse-4.3.0.tgz",
|
||||||
|
"integrity": "sha512-39nnKffWz8xN1BU/2c79n9nB9HDzo0niYUqx6xyqUnyoAnQyyWpOTdZEeiCch8BBu515t4wp9ZmgVfVhn9EBpw==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "BSD-2-Clause",
|
||||||
|
"peer": true,
|
||||||
|
"engines": {
|
||||||
|
"node": ">=4.0"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/webpack/node_modules/schema-utils": {
|
||||||
|
"version": "4.3.3",
|
||||||
|
"resolved": "https://registry.npmjs.org/schema-utils/-/schema-utils-4.3.3.tgz",
|
||||||
|
"integrity": "sha512-eflK8wEtyOE6+hsaRVPxvUKYCpRgzLqDTb8krvAsRIwOGlHoSgYLgBXoubGgLd2fT41/OUYdb48v4k4WWHQurA==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"peer": true,
|
||||||
|
"dependencies": {
|
||||||
|
"@types/json-schema": "^7.0.9",
|
||||||
|
"ajv": "^8.9.0",
|
||||||
|
"ajv-formats": "^2.1.1",
|
||||||
|
"ajv-keywords": "^5.1.0"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">= 10.13.0"
|
||||||
|
},
|
||||||
|
"funding": {
|
||||||
|
"type": "opencollective",
|
||||||
|
"url": "https://opencollective.com/webpack"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/whatwg-url": {
|
"node_modules/whatwg-url": {
|
||||||
"version": "5.0.0",
|
"version": "5.0.0",
|
||||||
"resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz",
|
"resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz",
|
||||||
|
|||||||
@@ -27,7 +27,7 @@
|
|||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@casl/ability": "^6.7.5",
|
"@casl/ability": "^6.7.5",
|
||||||
"@fastify/websocket": "^11.2.0",
|
"@fastify/websocket": "^10.0.1",
|
||||||
"@nestjs/bullmq": "^10.1.0",
|
"@nestjs/bullmq": "^10.1.0",
|
||||||
"@nestjs/common": "^10.3.0",
|
"@nestjs/common": "^10.3.0",
|
||||||
"@nestjs/config": "^3.1.1",
|
"@nestjs/config": "^3.1.1",
|
||||||
@@ -36,6 +36,7 @@
|
|||||||
"@nestjs/passport": "^10.0.3",
|
"@nestjs/passport": "^10.0.3",
|
||||||
"@nestjs/platform-fastify": "^10.3.0",
|
"@nestjs/platform-fastify": "^10.3.0",
|
||||||
"@nestjs/platform-socket.io": "^10.4.20",
|
"@nestjs/platform-socket.io": "^10.4.20",
|
||||||
|
"@nestjs/serve-static": "^4.0.2",
|
||||||
"@nestjs/websockets": "^10.4.20",
|
"@nestjs/websockets": "^10.4.20",
|
||||||
"@prisma/client": "^5.8.0",
|
"@prisma/client": "^5.8.0",
|
||||||
"bcrypt": "^5.1.1",
|
"bcrypt": "^5.1.1",
|
||||||
|
|||||||
@@ -3,13 +3,15 @@ import {
|
|||||||
FastifyAdapter,
|
FastifyAdapter,
|
||||||
NestFastifyApplication,
|
NestFastifyApplication,
|
||||||
} from '@nestjs/platform-fastify';
|
} from '@nestjs/platform-fastify';
|
||||||
import { ValidationPipe } from '@nestjs/common';
|
import { ValidationPipe, Logger } from '@nestjs/common';
|
||||||
import { AppModule } from './app.module';
|
import { AppModule } from './app.module';
|
||||||
|
import { VoiceService } from './voice/voice.service';
|
||||||
|
import { AudioConverterService } from './voice/audio-converter.service';
|
||||||
|
|
||||||
async function bootstrap() {
|
async function bootstrap() {
|
||||||
const app = await NestFactory.create<NestFastifyApplication>(
|
const app = await NestFactory.create<NestFastifyApplication>(
|
||||||
AppModule,
|
AppModule,
|
||||||
new FastifyAdapter({ logger: false }),
|
new FastifyAdapter({ logger: true }),
|
||||||
);
|
);
|
||||||
|
|
||||||
// Global validation pipe
|
// Global validation pipe
|
||||||
@@ -33,6 +35,144 @@ async function bootstrap() {
|
|||||||
const port = process.env.PORT || 3000;
|
const port = process.env.PORT || 3000;
|
||||||
await app.listen(port, '0.0.0.0');
|
await app.listen(port, '0.0.0.0');
|
||||||
|
|
||||||
|
// After app is listening, register WebSocket handler
|
||||||
|
const fastifyInstance = app.getHttpAdapter().getInstance();
|
||||||
|
const logger = new Logger('MediaStreamWS');
|
||||||
|
const voiceService = app.get(VoiceService);
|
||||||
|
const audioConverter = app.get(AudioConverterService);
|
||||||
|
|
||||||
|
const WebSocketServer = require('ws').Server;
|
||||||
|
const wss = new WebSocketServer({ noServer: true });
|
||||||
|
|
||||||
|
// Handle WebSocket upgrades at the server level
|
||||||
|
const server = (fastifyInstance.server as any);
|
||||||
|
|
||||||
|
// Track active Media Streams connections: streamSid -> WebSocket
|
||||||
|
const mediaStreams: Map<string, any> = new Map();
|
||||||
|
|
||||||
|
server.on('upgrade', (request: any, socket: any, head: any) => {
|
||||||
|
if (request.url === '/api/voice/media-stream') {
|
||||||
|
logger.log('=== MEDIA STREAM WEBSOCKET UPGRADE REQUEST ===');
|
||||||
|
logger.log(`Path: ${request.url}`);
|
||||||
|
|
||||||
|
wss.handleUpgrade(request, socket, head, (ws: any) => {
|
||||||
|
logger.log('=== MEDIA STREAM WEBSOCKET UPGRADED SUCCESSFULLY ===');
|
||||||
|
handleMediaStreamSocket(ws);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
async function handleMediaStreamSocket(ws: any) {
|
||||||
|
let streamSid: string | null = null;
|
||||||
|
let callSid: string | null = null;
|
||||||
|
let tenantDomain: string | null = null;
|
||||||
|
let mediaPacketCount = 0;
|
||||||
|
|
||||||
|
ws.on('message', async (message: Buffer) => {
|
||||||
|
try {
|
||||||
|
const msg = JSON.parse(message.toString());
|
||||||
|
|
||||||
|
switch (msg.event) {
|
||||||
|
case 'connected':
|
||||||
|
logger.log('=== MEDIA STREAM EVENT: CONNECTED ===');
|
||||||
|
logger.log(`Protocol: ${msg.protocol}`);
|
||||||
|
logger.log(`Version: ${msg.version}`);
|
||||||
|
break;
|
||||||
|
|
||||||
|
case 'start':
|
||||||
|
streamSid = msg.streamSid;
|
||||||
|
callSid = msg.start.callSid;
|
||||||
|
tenantDomain = msg.start.customParameters?.tenantId || 'tenant1';
|
||||||
|
|
||||||
|
logger.log(`=== MEDIA STREAM EVENT: START ===`);
|
||||||
|
logger.log(`StreamSid: ${streamSid}`);
|
||||||
|
logger.log(`CallSid: ${callSid}`);
|
||||||
|
logger.log(`Tenant: ${tenantDomain}`);
|
||||||
|
logger.log(`MediaFormat: ${JSON.stringify(msg.start.mediaFormat)}`);
|
||||||
|
|
||||||
|
mediaStreams.set(streamSid, ws);
|
||||||
|
logger.log(`Stored WebSocket for streamSid: ${streamSid}. Total active streams: ${mediaStreams.size}`);
|
||||||
|
|
||||||
|
// Initialize OpenAI Realtime connection
|
||||||
|
logger.log(`Initializing OpenAI Realtime for call ${callSid}...`);
|
||||||
|
try {
|
||||||
|
await voiceService.initializeOpenAIRealtime({
|
||||||
|
callSid,
|
||||||
|
tenantId: tenantDomain,
|
||||||
|
userId: msg.start.customParameters?.userId || 'system',
|
||||||
|
});
|
||||||
|
logger.log(`✓ OpenAI Realtime initialized for call ${callSid}`);
|
||||||
|
} catch (error: any) {
|
||||||
|
logger.error(`Failed to initialize OpenAI: ${error.message}`);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
|
||||||
|
case 'media':
|
||||||
|
mediaPacketCount++;
|
||||||
|
if (mediaPacketCount % 50 === 0) {
|
||||||
|
logger.log(`Received media packet #${mediaPacketCount} for StreamSid: ${streamSid}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!callSid || !tenantDomain) {
|
||||||
|
logger.warn('Received media before start event');
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Convert Twilio audio (μ-law 8kHz) to OpenAI format (PCM16 24kHz)
|
||||||
|
const twilioAudio = msg.media.payload;
|
||||||
|
const openaiAudio = audioConverter.twilioToOpenAI(twilioAudio);
|
||||||
|
|
||||||
|
// Send audio to OpenAI Realtime API
|
||||||
|
await voiceService.sendAudioToOpenAI(callSid, openaiAudio);
|
||||||
|
} catch (error: any) {
|
||||||
|
logger.error(`Error processing media: ${error.message}`);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
|
||||||
|
case 'stop':
|
||||||
|
logger.log(`=== MEDIA STREAM EVENT: STOP ===`);
|
||||||
|
logger.log(`StreamSid: ${streamSid}`);
|
||||||
|
logger.log(`Total media packets received: ${mediaPacketCount}`);
|
||||||
|
|
||||||
|
if (streamSid) {
|
||||||
|
mediaStreams.delete(streamSid);
|
||||||
|
logger.log(`Removed WebSocket for streamSid: ${streamSid}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Clean up OpenAI connection
|
||||||
|
if (callSid) {
|
||||||
|
try {
|
||||||
|
logger.log(`Cleaning up OpenAI connection for call ${callSid}...`);
|
||||||
|
await voiceService.cleanupOpenAIConnection(callSid);
|
||||||
|
logger.log(`✓ OpenAI connection cleaned up`);
|
||||||
|
} catch (error: any) {
|
||||||
|
logger.error(`Failed to cleanup OpenAI: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
|
||||||
|
default:
|
||||||
|
logger.debug(`Unknown media stream event: ${msg.event}`);
|
||||||
|
}
|
||||||
|
} catch (error: any) {
|
||||||
|
logger.error(`Error processing media stream message: ${error.message}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
ws.on('close', () => {
|
||||||
|
logger.log(`=== MEDIA STREAM WEBSOCKET CLOSED ===`);
|
||||||
|
if (streamSid) {
|
||||||
|
mediaStreams.delete(streamSid);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
ws.on('error', (error: Error) => {
|
||||||
|
logger.error(`=== MEDIA STREAM WEBSOCKET ERROR ===`);
|
||||||
|
logger.error(`Error message: ${error.message}`);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
console.log(`🚀 Application is running on: http://localhost:${port}/api`);
|
console.log(`🚀 Application is running on: http://localhost:${port}/api`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
214
backend/src/voice/audio-converter.service.ts
Normal file
214
backend/src/voice/audio-converter.service.ts
Normal file
@@ -0,0 +1,214 @@
|
|||||||
|
import { Injectable, Logger } from '@nestjs/common';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Audio format converter for Twilio <-> OpenAI audio streaming
|
||||||
|
*
|
||||||
|
* Twilio Media Streams format:
|
||||||
|
* - Codec: μ-law (G.711)
|
||||||
|
* - Sample rate: 8kHz
|
||||||
|
* - Encoding: base64
|
||||||
|
* - Chunk size: 20ms (160 bytes)
|
||||||
|
*
|
||||||
|
* OpenAI Realtime API format:
|
||||||
|
* - Codec: PCM16
|
||||||
|
* - Sample rate: 24kHz
|
||||||
|
* - Encoding: base64
|
||||||
|
* - Mono channel
|
||||||
|
*/
|
||||||
|
@Injectable()
|
||||||
|
export class AudioConverterService {
|
||||||
|
private readonly logger = new Logger(AudioConverterService.name);
|
||||||
|
|
||||||
|
// μ-law decode lookup table
|
||||||
|
private readonly MULAW_DECODE_TABLE = this.buildMuLawDecodeTable();
|
||||||
|
|
||||||
|
// μ-law encode lookup table
|
||||||
|
private readonly MULAW_ENCODE_TABLE = this.buildMuLawEncodeTable();
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Build μ-law to linear PCM16 decode table
|
||||||
|
*/
|
||||||
|
private buildMuLawDecodeTable(): Int16Array {
|
||||||
|
const table = new Int16Array(256);
|
||||||
|
for (let i = 0; i < 256; i++) {
|
||||||
|
const mulaw = ~i;
|
||||||
|
const exponent = (mulaw >> 4) & 0x07;
|
||||||
|
const mantissa = mulaw & 0x0f;
|
||||||
|
let sample = (mantissa << 3) + 0x84;
|
||||||
|
sample <<= exponent;
|
||||||
|
sample -= 0x84;
|
||||||
|
if ((mulaw & 0x80) === 0) {
|
||||||
|
sample = -sample;
|
||||||
|
}
|
||||||
|
table[i] = sample;
|
||||||
|
}
|
||||||
|
return table;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Build linear PCM16 to μ-law encode table
|
||||||
|
*/
|
||||||
|
private buildMuLawEncodeTable(): Uint8Array {
|
||||||
|
const table = new Uint8Array(65536);
|
||||||
|
for (let i = 0; i < 65536; i++) {
|
||||||
|
const sample = (i - 32768);
|
||||||
|
const sign = sample < 0 ? 0x80 : 0x00;
|
||||||
|
const magnitude = Math.abs(sample);
|
||||||
|
|
||||||
|
// Add bias
|
||||||
|
let biased = magnitude + 0x84;
|
||||||
|
|
||||||
|
// Find exponent
|
||||||
|
let exponent = 7;
|
||||||
|
for (let exp = 0; exp < 8; exp++) {
|
||||||
|
if (biased <= (0xff << exp)) {
|
||||||
|
exponent = exp;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract mantissa
|
||||||
|
const mantissa = (biased >> (exponent + 3)) & 0x0f;
|
||||||
|
|
||||||
|
// Combine sign, exponent, mantissa
|
||||||
|
const mulaw = ~(sign | (exponent << 4) | mantissa);
|
||||||
|
table[i] = mulaw & 0xff;
|
||||||
|
}
|
||||||
|
return table;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Decode μ-law audio to linear PCM16
|
||||||
|
* @param mulawData - Buffer containing μ-law encoded audio
|
||||||
|
* @returns Buffer containing PCM16 audio (16-bit little-endian)
|
||||||
|
*/
|
||||||
|
decodeMuLaw(mulawData: Buffer): Buffer {
|
||||||
|
const pcm16 = Buffer.allocUnsafe(mulawData.length * 2);
|
||||||
|
|
||||||
|
for (let i = 0; i < mulawData.length; i++) {
|
||||||
|
const sample = this.MULAW_DECODE_TABLE[mulawData[i]];
|
||||||
|
pcm16.writeInt16LE(sample, i * 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
return pcm16;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Encode linear PCM16 to μ-law
|
||||||
|
* @param pcm16Data - Buffer containing PCM16 audio (16-bit little-endian)
|
||||||
|
* @returns Buffer containing μ-law encoded audio
|
||||||
|
*/
|
||||||
|
encodeMuLaw(pcm16Data: Buffer): Buffer {
|
||||||
|
const mulaw = Buffer.allocUnsafe(pcm16Data.length / 2);
|
||||||
|
|
||||||
|
for (let i = 0; i < pcm16Data.length; i += 2) {
|
||||||
|
const sample = pcm16Data.readInt16LE(i);
|
||||||
|
const index = (sample + 32768) & 0xffff;
|
||||||
|
mulaw[i / 2] = this.MULAW_ENCODE_TABLE[index];
|
||||||
|
}
|
||||||
|
|
||||||
|
return mulaw;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resample audio from 8kHz to 24kHz (linear interpolation)
|
||||||
|
* @param pcm16Data - Buffer containing 8kHz PCM16 audio
|
||||||
|
* @returns Buffer containing 24kHz PCM16 audio
|
||||||
|
*/
|
||||||
|
resample8kTo24k(pcm16Data: Buffer): Buffer {
|
||||||
|
const inputSamples = pcm16Data.length / 2;
|
||||||
|
const outputSamples = Math.floor(inputSamples * 3); // 8k * 3 = 24k
|
||||||
|
const output = Buffer.allocUnsafe(outputSamples * 2);
|
||||||
|
|
||||||
|
for (let i = 0; i < outputSamples; i++) {
|
||||||
|
const srcIndex = i / 3;
|
||||||
|
const srcIndexFloor = Math.floor(srcIndex);
|
||||||
|
const srcIndexCeil = Math.min(srcIndexFloor + 1, inputSamples - 1);
|
||||||
|
const fraction = srcIndex - srcIndexFloor;
|
||||||
|
|
||||||
|
const sample1 = pcm16Data.readInt16LE(srcIndexFloor * 2);
|
||||||
|
const sample2 = pcm16Data.readInt16LE(srcIndexCeil * 2);
|
||||||
|
|
||||||
|
// Linear interpolation
|
||||||
|
const interpolated = Math.round(sample1 + (sample2 - sample1) * fraction);
|
||||||
|
output.writeInt16LE(interpolated, i * 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
return output;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resample audio from 24kHz to 8kHz (decimation with averaging)
|
||||||
|
* @param pcm16Data - Buffer containing 24kHz PCM16 audio
|
||||||
|
* @returns Buffer containing 8kHz PCM16 audio
|
||||||
|
*/
|
||||||
|
resample24kTo8k(pcm16Data: Buffer): Buffer {
|
||||||
|
const inputSamples = pcm16Data.length / 2;
|
||||||
|
const outputSamples = Math.floor(inputSamples / 3); // 24k / 3 = 8k
|
||||||
|
const output = Buffer.allocUnsafe(outputSamples * 2);
|
||||||
|
|
||||||
|
for (let i = 0; i < outputSamples; i++) {
|
||||||
|
// Average 3 samples for anti-aliasing
|
||||||
|
const idx1 = Math.min(i * 3, inputSamples - 1);
|
||||||
|
const idx2 = Math.min(i * 3 + 1, inputSamples - 1);
|
||||||
|
const idx3 = Math.min(i * 3 + 2, inputSamples - 1);
|
||||||
|
|
||||||
|
const sample1 = pcm16Data.readInt16LE(idx1 * 2);
|
||||||
|
const sample2 = pcm16Data.readInt16LE(idx2 * 2);
|
||||||
|
const sample3 = pcm16Data.readInt16LE(idx3 * 2);
|
||||||
|
|
||||||
|
const averaged = Math.round((sample1 + sample2 + sample3) / 3);
|
||||||
|
output.writeInt16LE(averaged, i * 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
return output;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Convert Twilio μ-law 8kHz to OpenAI PCM16 24kHz
|
||||||
|
* @param twilioBase64 - Base64-encoded μ-law audio from Twilio
|
||||||
|
* @returns Base64-encoded PCM16 24kHz audio for OpenAI
|
||||||
|
*/
|
||||||
|
twilioToOpenAI(twilioBase64: string): string {
|
||||||
|
try {
|
||||||
|
// Decode base64
|
||||||
|
const mulawBuffer = Buffer.from(twilioBase64, 'base64');
|
||||||
|
|
||||||
|
// μ-law -> PCM16
|
||||||
|
const pcm16_8k = this.decodeMuLaw(mulawBuffer);
|
||||||
|
|
||||||
|
// 8kHz -> 24kHz
|
||||||
|
const pcm16_24k = this.resample8kTo24k(pcm16_8k);
|
||||||
|
|
||||||
|
// Encode to base64
|
||||||
|
return pcm16_24k.toString('base64');
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.error('Error converting Twilio to OpenAI audio', error);
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Convert OpenAI PCM16 24kHz to Twilio μ-law 8kHz
|
||||||
|
* @param openaiBase64 - Base64-encoded PCM16 24kHz audio from OpenAI
|
||||||
|
* @returns Base64-encoded μ-law 8kHz audio for Twilio
|
||||||
|
*/
|
||||||
|
openAIToTwilio(openaiBase64: string): string {
|
||||||
|
try {
|
||||||
|
// Decode base64
|
||||||
|
const pcm16_24k = Buffer.from(openaiBase64, 'base64');
|
||||||
|
|
||||||
|
// 24kHz -> 8kHz
|
||||||
|
const pcm16_8k = this.resample24kTo8k(pcm16_24k);
|
||||||
|
|
||||||
|
// PCM16 -> μ-law
|
||||||
|
const mulawBuffer = this.encodeMuLaw(pcm16_8k);
|
||||||
|
|
||||||
|
// Encode to base64
|
||||||
|
return mulawBuffer.toString('base64');
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.error('Error converting OpenAI to Twilio audio', error);
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -13,6 +13,7 @@ import { FastifyRequest, FastifyReply } from 'fastify';
|
|||||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
import { JwtAuthGuard } from '../auth/jwt-auth.guard';
|
||||||
import { VoiceService } from './voice.service';
|
import { VoiceService } from './voice.service';
|
||||||
import { VoiceGateway } from './voice.gateway';
|
import { VoiceGateway } from './voice.gateway';
|
||||||
|
import { AudioConverterService } from './audio-converter.service';
|
||||||
import { InitiateCallDto } from './dto/initiate-call.dto';
|
import { InitiateCallDto } from './dto/initiate-call.dto';
|
||||||
import { TenantId } from '../tenant/tenant.decorator';
|
import { TenantId } from '../tenant/tenant.decorator';
|
||||||
|
|
||||||
@@ -20,9 +21,13 @@ import { TenantId } from '../tenant/tenant.decorator';
|
|||||||
export class VoiceController {
|
export class VoiceController {
|
||||||
private readonly logger = new Logger(VoiceController.name);
|
private readonly logger = new Logger(VoiceController.name);
|
||||||
|
|
||||||
|
// Track active Media Streams connections: streamSid -> WebSocket
|
||||||
|
private mediaStreams: Map<string, any> = new Map();
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
private readonly voiceService: VoiceService,
|
private readonly voiceService: VoiceService,
|
||||||
private readonly voiceGateway: VoiceGateway,
|
private readonly voiceGateway: VoiceGateway,
|
||||||
|
private readonly audioConverter: AudioConverterService,
|
||||||
) {}
|
) {}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -159,8 +164,12 @@ export class VoiceController {
|
|||||||
const fromNumber = body.From;
|
const fromNumber = body.From;
|
||||||
const toNumber = body.To;
|
const toNumber = body.To;
|
||||||
|
|
||||||
this.logger.log(`=== INBOUND CALL RECEIVED ===`);
|
this.logger.log(`\n\n╔════════════════════════════════════════╗`);
|
||||||
this.logger.log(`CallSid: ${callSid}, From: ${fromNumber}, To: ${toNumber}`);
|
this.logger.log(`║ === INBOUND CALL RECEIVED ===`);
|
||||||
|
this.logger.log(`╚════════════════════════════════════════╝`);
|
||||||
|
this.logger.log(`CallSid: ${callSid}`);
|
||||||
|
this.logger.log(`From: ${fromNumber}`);
|
||||||
|
this.logger.log(`To: ${toNumber}`);
|
||||||
this.logger.log(`Full body: ${JSON.stringify(body)}`);
|
this.logger.log(`Full body: ${JSON.stringify(body)}`);
|
||||||
|
|
||||||
try {
|
try {
|
||||||
@@ -174,6 +183,9 @@ export class VoiceController {
|
|||||||
const connectedUsers = this.voiceGateway.getConnectedUsers(tenantDomain);
|
const connectedUsers = this.voiceGateway.getConnectedUsers(tenantDomain);
|
||||||
|
|
||||||
this.logger.log(`Connected users for tenant ${tenantDomain}: ${connectedUsers.length}`);
|
this.logger.log(`Connected users for tenant ${tenantDomain}: ${connectedUsers.length}`);
|
||||||
|
if (connectedUsers.length > 0) {
|
||||||
|
this.logger.log(`Connected user IDs: ${connectedUsers.join(', ')}`);
|
||||||
|
}
|
||||||
|
|
||||||
if (connectedUsers.length === 0) {
|
if (connectedUsers.length === 0) {
|
||||||
// No users online - send to voicemail or play message
|
// No users online - send to voicemail or play message
|
||||||
@@ -182,22 +194,52 @@ export class VoiceController {
|
|||||||
<Say>Sorry, no agents are currently available. Please try again later.</Say>
|
<Say>Sorry, no agents are currently available. Please try again later.</Say>
|
||||||
<Hangup/>
|
<Hangup/>
|
||||||
</Response>`;
|
</Response>`;
|
||||||
this.logger.log(`No users online - returning unavailable message`);
|
this.logger.log(`❌ No users online - returning unavailable message`);
|
||||||
return res.type('text/xml').send(twiml);
|
return res.type('text/xml').send(twiml);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Build TwiML to dial all connected clients (first to answer gets the call)
|
// Build TwiML to dial all connected clients with Media Streams for AI
|
||||||
const clientElements = connectedUsers.map(userId => ` <Client>${userId}</Client>`).join('\n');
|
const clientElements = connectedUsers.map(userId => ` <Client>${userId}</Client>`).join('\n');
|
||||||
|
|
||||||
|
// Use wss:// for secure WebSocket (Traefik handles HTTPS)
|
||||||
|
const streamUrl = `wss://${host}/api/voice/media-stream`;
|
||||||
|
|
||||||
|
this.logger.log(`Stream URL: ${streamUrl}`);
|
||||||
|
this.logger.log(`Dialing ${connectedUsers.length} client(s)...`);
|
||||||
|
this.logger.log(`Client IDs to dial: ${connectedUsers.join(', ')}`);
|
||||||
|
|
||||||
|
// Verify we have client IDs in proper format
|
||||||
|
if (connectedUsers.length > 0) {
|
||||||
|
this.logger.log(`First Client ID format check: "${connectedUsers[0]}" (length: ${connectedUsers[0].length})`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Notify connected users about incoming call via Socket.IO
|
||||||
|
connectedUsers.forEach(userId => {
|
||||||
|
this.voiceGateway.notifyIncomingCall(userId, {
|
||||||
|
callSid,
|
||||||
|
fromNumber,
|
||||||
|
toNumber,
|
||||||
|
tenantDomain,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
const twiml = `<?xml version="1.0" encoding="UTF-8"?>
|
const twiml = `<?xml version="1.0" encoding="UTF-8"?>
|
||||||
<Response>
|
<Response>
|
||||||
<Say>Connecting your call</Say>
|
<Dial timeout="30">
|
||||||
<Dial timeout="30" action="${host}/api/voice/webhook/dial-status">
|
|
||||||
${clientElements}
|
${clientElements}
|
||||||
|
<OnAnswer>
|
||||||
|
<Connect>
|
||||||
|
<Stream url="${streamUrl}">
|
||||||
|
<Parameter name="tenantId" value="${tenantDomain}"/>
|
||||||
|
<Parameter name="userId" value="${connectedUsers[0]}"/>
|
||||||
|
</Stream>
|
||||||
|
</Connect>
|
||||||
|
</OnAnswer>
|
||||||
</Dial>
|
</Dial>
|
||||||
</Response>`;
|
</Response>`;
|
||||||
|
|
||||||
this.logger.log(`Returning inbound TwiML - dialing ${connectedUsers.length} client(s)`);
|
this.logger.log(`✓ Returning inbound TwiML with Media Streams - dialing ${connectedUsers.length} client(s)`);
|
||||||
|
this.logger.log(`Generated TwiML:\n${twiml}\n`);
|
||||||
res.type('text/xml').send(twiml);
|
res.type('text/xml').send(twiml);
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
this.logger.error(`Error generating inbound TwiML: ${error.message}`);
|
this.logger.error(`Error generating inbound TwiML: ${error.message}`);
|
||||||
@@ -240,4 +282,216 @@ ${clientElements}
|
|||||||
|
|
||||||
return { success: true };
|
return { success: true };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Twilio Media Streams WebSocket endpoint
|
||||||
|
* Receives real-time audio from Twilio and forwards to OpenAI Realtime API
|
||||||
|
*
|
||||||
|
* This handles the HTTP GET request and upgrades it to WebSocket manually.
|
||||||
|
*/
|
||||||
|
@Get('media-stream')
|
||||||
|
mediaStream(@Req() req: FastifyRequest) {
|
||||||
|
// For WebSocket upgrade, we need to access the raw socket
|
||||||
|
let socket: any;
|
||||||
|
|
||||||
|
try {
|
||||||
|
this.logger.log(`=== MEDIA STREAM REQUEST ===`);
|
||||||
|
this.logger.log(`URL: ${req.url}`);
|
||||||
|
this.logger.log(`Headers keys: ${Object.keys(req.headers).join(', ')}`);
|
||||||
|
this.logger.log(`Headers: ${JSON.stringify(req.headers)}`);
|
||||||
|
|
||||||
|
// Check if this is a WebSocket upgrade request
|
||||||
|
const hasWebSocketKey = 'sec-websocket-key' in req.headers;
|
||||||
|
const hasWebSocketVersion = 'sec-websocket-version' in req.headers;
|
||||||
|
|
||||||
|
this.logger.log(`hasWebSocketKey: ${hasWebSocketKey}`);
|
||||||
|
this.logger.log(`hasWebSocketVersion: ${hasWebSocketVersion}`);
|
||||||
|
|
||||||
|
if (!hasWebSocketKey || !hasWebSocketVersion) {
|
||||||
|
this.logger.log('Not a WebSocket upgrade request - returning');
|
||||||
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
this.logger.log('✓ WebSocket upgrade detected');
|
||||||
|
|
||||||
|
// Get the socket - try different ways
|
||||||
|
socket = (req.raw as any).socket;
|
||||||
|
this.logger.log(`Socket obtained: ${!!socket}`);
|
||||||
|
|
||||||
|
if (!socket) {
|
||||||
|
this.logger.error('Failed to get socket from req.raw');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const rawRequest = req.raw;
|
||||||
|
const head = Buffer.alloc(0);
|
||||||
|
|
||||||
|
this.logger.log('Creating WebSocketServer...');
|
||||||
|
const WebSocketServer = require('ws').Server;
|
||||||
|
const wss = new WebSocketServer({ noServer: true });
|
||||||
|
|
||||||
|
this.logger.log('Calling handleUpgrade...');
|
||||||
|
|
||||||
|
// handleUpgrade will send the 101 response and take over the socket
|
||||||
|
wss.handleUpgrade(rawRequest, socket, head, (ws: any) => {
|
||||||
|
this.logger.log('=== TWILIO MEDIA STREAM WEBSOCKET UPGRADED SUCCESSFULLY ===');
|
||||||
|
this.handleMediaStreamSocket(ws);
|
||||||
|
});
|
||||||
|
|
||||||
|
this.logger.log('handleUpgrade completed');
|
||||||
|
} catch (error: any) {
|
||||||
|
this.logger.error(`=== FAILED TO UPGRADE TO WEBSOCKET ===`);
|
||||||
|
this.logger.error(`Error message: ${error.message}`);
|
||||||
|
this.logger.error(`Error stack: ${error.stack}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Handle incoming Media Stream WebSocket messages
|
||||||
|
*/
|
||||||
|
private handleMediaStreamSocket(ws: any) {
|
||||||
|
let streamSid: string | null = null;
|
||||||
|
let callSid: string | null = null;
|
||||||
|
let tenantDomain: string | null = null;
|
||||||
|
let mediaPacketCount = 0;
|
||||||
|
|
||||||
|
// WebSocket message handler
|
||||||
|
ws.on('message', async (message: Buffer) => {
|
||||||
|
try {
|
||||||
|
const msg = JSON.parse(message.toString());
|
||||||
|
|
||||||
|
switch (msg.event) {
|
||||||
|
case 'connected':
|
||||||
|
this.logger.log('=== MEDIA STREAM EVENT: CONNECTED ===');
|
||||||
|
this.logger.log(`Protocol: ${msg.protocol}`);
|
||||||
|
this.logger.log(`Version: ${msg.version}`);
|
||||||
|
break;
|
||||||
|
|
||||||
|
case 'start':
|
||||||
|
streamSid = msg.streamSid;
|
||||||
|
callSid = msg.start.callSid;
|
||||||
|
|
||||||
|
// Extract tenant from customParameters if available
|
||||||
|
tenantDomain = msg.start.customParameters?.tenantId || 'tenant1';
|
||||||
|
|
||||||
|
this.logger.log(`=== MEDIA STREAM EVENT: START ===`);
|
||||||
|
this.logger.log(`StreamSid: ${streamSid}`);
|
||||||
|
this.logger.log(`CallSid: ${callSid}`);
|
||||||
|
this.logger.log(`Tenant: ${tenantDomain}`);
|
||||||
|
this.logger.log(`AccountSid: ${msg.start.accountSid}`);
|
||||||
|
this.logger.log(`MediaFormat: ${JSON.stringify(msg.start.mediaFormat)}`);
|
||||||
|
this.logger.log(`Custom Parameters: ${JSON.stringify(msg.start.customParameters)}`);
|
||||||
|
|
||||||
|
// Store WebSocket connection
|
||||||
|
this.mediaStreams.set(streamSid, ws);
|
||||||
|
this.logger.log(`Stored WebSocket for streamSid: ${streamSid}. Total active streams: ${this.mediaStreams.size}`);
|
||||||
|
|
||||||
|
// Initialize OpenAI Realtime connection for this call
|
||||||
|
this.logger.log(`Initializing OpenAI Realtime for call ${callSid}...`);
|
||||||
|
await this.voiceService.initializeOpenAIRealtime({
|
||||||
|
callSid,
|
||||||
|
tenantId: tenantDomain,
|
||||||
|
userId: msg.start.customParameters?.userId || 'system',
|
||||||
|
});
|
||||||
|
|
||||||
|
this.logger.log(`✓ OpenAI Realtime initialized for call ${callSid}`);
|
||||||
|
break;
|
||||||
|
|
||||||
|
case 'media':
|
||||||
|
mediaPacketCount++;
|
||||||
|
if (mediaPacketCount % 50 === 0) {
|
||||||
|
// Log every 50th packet to avoid spam
|
||||||
|
this.logger.log(`Received media packet #${mediaPacketCount} for StreamSid: ${streamSid}, CallSid: ${callSid}, PayloadSize: ${msg.media.payload?.length || 0} bytes`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!callSid || !tenantDomain) {
|
||||||
|
this.logger.warn('Received media before start event');
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
// msg.media.payload is base64-encoded μ-law audio from Twilio
|
||||||
|
const twilioAudio = msg.media.payload;
|
||||||
|
|
||||||
|
// Convert Twilio audio (μ-law 8kHz) to OpenAI format (PCM16 24kHz)
|
||||||
|
const openaiAudio = this.audioConverter.twilioToOpenAI(twilioAudio);
|
||||||
|
|
||||||
|
// Send audio to OpenAI Realtime API
|
||||||
|
await this.voiceService.sendAudioToOpenAI(callSid, openaiAudio);
|
||||||
|
break;
|
||||||
|
|
||||||
|
case 'stop':
|
||||||
|
this.logger.log(`=== MEDIA STREAM EVENT: STOP ===`);
|
||||||
|
this.logger.log(`StreamSid: ${streamSid}`);
|
||||||
|
this.logger.log(`Total media packets received: ${mediaPacketCount}`);
|
||||||
|
|
||||||
|
if (streamSid) {
|
||||||
|
this.mediaStreams.delete(streamSid);
|
||||||
|
this.logger.log(`Removed WebSocket for streamSid: ${streamSid}. Remaining active streams: ${this.mediaStreams.size}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Clean up OpenAI connection
|
||||||
|
if (callSid) {
|
||||||
|
this.logger.log(`Cleaning up OpenAI connection for call ${callSid}...`);
|
||||||
|
await this.voiceService.cleanupOpenAIConnection(callSid);
|
||||||
|
this.logger.log(`✓ OpenAI connection cleaned up for call ${callSid}`);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
|
||||||
|
default:
|
||||||
|
this.logger.debug(`Unknown media stream event: ${msg.event}`);
|
||||||
|
}
|
||||||
|
} catch (error: any) {
|
||||||
|
this.logger.error(`Error processing media stream message: ${error.message}`);
|
||||||
|
this.logger.error(`Stack: ${error.stack}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
ws.on('close', () => {
|
||||||
|
this.logger.log(`=== MEDIA STREAM WEBSOCKET CLOSED ===`);
|
||||||
|
this.logger.log(`StreamSid: ${streamSid}`);
|
||||||
|
this.logger.log(`Total media packets in this stream: ${mediaPacketCount}`);
|
||||||
|
if (streamSid) {
|
||||||
|
this.mediaStreams.delete(streamSid);
|
||||||
|
this.logger.log(`Cleaned up streamSid on close. Remaining active streams: ${this.mediaStreams.size}`);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
ws.on('error', (error: Error) => {
|
||||||
|
this.logger.error(`=== MEDIA STREAM WEBSOCKET ERROR ===`);
|
||||||
|
this.logger.error(`StreamSid: ${streamSid}`);
|
||||||
|
this.logger.error(`Error message: ${error.message}`);
|
||||||
|
this.logger.error(`Error stack: ${error.stack}`);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Send audio from OpenAI back to Twilio Media Stream
|
||||||
|
*/
|
||||||
|
async sendAudioToTwilio(streamSid: string, openaiAudioBase64: string) {
|
||||||
|
const ws = this.mediaStreams.get(streamSid);
|
||||||
|
|
||||||
|
if (!ws) {
|
||||||
|
this.logger.warn(`No Media Stream found for streamSid: ${streamSid}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Convert OpenAI audio (PCM16 24kHz) to Twilio format (μ-law 8kHz)
|
||||||
|
const twilioAudio = this.audioConverter.openAIToTwilio(openaiAudioBase64);
|
||||||
|
|
||||||
|
// Send to Twilio Media Stream
|
||||||
|
const message = {
|
||||||
|
event: 'media',
|
||||||
|
streamSid,
|
||||||
|
media: {
|
||||||
|
payload: twilioAudio,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
ws.send(JSON.stringify(message));
|
||||||
|
} catch (error: any) {
|
||||||
|
this.logger.error(`Error sending audio to Twilio: ${error.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -40,7 +40,10 @@ export class VoiceGateway
|
|||||||
private readonly jwtService: JwtService,
|
private readonly jwtService: JwtService,
|
||||||
private readonly voiceService: VoiceService,
|
private readonly voiceService: VoiceService,
|
||||||
private readonly tenantDbService: TenantDatabaseService,
|
private readonly tenantDbService: TenantDatabaseService,
|
||||||
) {}
|
) {
|
||||||
|
// Set gateway reference in service to avoid circular dependency
|
||||||
|
this.voiceService.setGateway(this);
|
||||||
|
}
|
||||||
|
|
||||||
async handleConnection(client: AuthenticatedSocket) {
|
async handleConnection(client: AuthenticatedSocket) {
|
||||||
try {
|
try {
|
||||||
@@ -49,7 +52,7 @@ export class VoiceGateway
|
|||||||
client.handshake.auth.token || client.handshake.headers.authorization?.split(' ')[1];
|
client.handshake.auth.token || client.handshake.headers.authorization?.split(' ')[1];
|
||||||
|
|
||||||
if (!token) {
|
if (!token) {
|
||||||
this.logger.warn('Client connection rejected: No token provided');
|
this.logger.warn('❌ Client connection rejected: No token provided');
|
||||||
client.disconnect();
|
client.disconnect();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -82,8 +85,9 @@ export class VoiceGateway
|
|||||||
|
|
||||||
this.connectedUsers.set(client.userId, client);
|
this.connectedUsers.set(client.userId, client);
|
||||||
this.logger.log(
|
this.logger.log(
|
||||||
`Client connected: ${client.id} (User: ${client.userId}, Domain: ${domain})`,
|
`✓ Client connected: ${client.id} (User: ${client.userId}, Domain: ${domain})`,
|
||||||
);
|
);
|
||||||
|
this.logger.log(`Total connected users in ${domain}: ${this.getConnectedUsers(domain).length}`);
|
||||||
|
|
||||||
// Send current call state if any active call
|
// Send current call state if any active call
|
||||||
const activeCallSid = this.activeCallsByUser.get(client.userId);
|
const activeCallSid = this.activeCallsByUser.get(client.userId);
|
||||||
@@ -95,7 +99,7 @@ export class VoiceGateway
|
|||||||
client.emit('call:state', callState);
|
client.emit('call:state', callState);
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
this.logger.error('Authentication failed', error);
|
this.logger.error('❌ Authentication failed', error);
|
||||||
client.disconnect();
|
client.disconnect();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -103,7 +107,8 @@ export class VoiceGateway
|
|||||||
handleDisconnect(client: AuthenticatedSocket) {
|
handleDisconnect(client: AuthenticatedSocket) {
|
||||||
if (client.userId) {
|
if (client.userId) {
|
||||||
this.connectedUsers.delete(client.userId);
|
this.connectedUsers.delete(client.userId);
|
||||||
this.logger.log(`Client disconnected: ${client.id} (User: ${client.userId})`);
|
this.logger.log(`✓ Client disconnected: ${client.id} (User: ${client.userId})`);
|
||||||
|
this.logger.log(`Remaining connected users: ${this.connectedUsers.size}`);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import { JwtModule } from '@nestjs/jwt';
|
|||||||
import { VoiceGateway } from './voice.gateway';
|
import { VoiceGateway } from './voice.gateway';
|
||||||
import { VoiceService } from './voice.service';
|
import { VoiceService } from './voice.service';
|
||||||
import { VoiceController } from './voice.controller';
|
import { VoiceController } from './voice.controller';
|
||||||
|
import { AudioConverterService } from './audio-converter.service';
|
||||||
import { TenantModule } from '../tenant/tenant.module';
|
import { TenantModule } from '../tenant/tenant.module';
|
||||||
import { AuthModule } from '../auth/auth.module';
|
import { AuthModule } from '../auth/auth.module';
|
||||||
|
|
||||||
@@ -15,7 +16,7 @@ import { AuthModule } from '../auth/auth.module';
|
|||||||
signOptions: { expiresIn: process.env.JWT_EXPIRES_IN || '24h' },
|
signOptions: { expiresIn: process.env.JWT_EXPIRES_IN || '24h' },
|
||||||
}),
|
}),
|
||||||
],
|
],
|
||||||
providers: [VoiceGateway, VoiceService],
|
providers: [VoiceGateway, VoiceService, AudioConverterService],
|
||||||
controllers: [VoiceController],
|
controllers: [VoiceController],
|
||||||
exports: [VoiceService],
|
exports: [VoiceService],
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -15,11 +15,19 @@ export class VoiceService {
|
|||||||
private twilioClients: Map<string, Twilio.Twilio> = new Map();
|
private twilioClients: Map<string, Twilio.Twilio> = new Map();
|
||||||
private openaiConnections: Map<string, WebSocket> = new Map(); // callSid -> WebSocket
|
private openaiConnections: Map<string, WebSocket> = new Map(); // callSid -> WebSocket
|
||||||
private callStates: Map<string, any> = new Map(); // callSid -> call state
|
private callStates: Map<string, any> = new Map(); // callSid -> call state
|
||||||
|
private voiceGateway: any; // Reference to gateway (to avoid circular dependency)
|
||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
private readonly tenantDbService: TenantDatabaseService,
|
private readonly tenantDbService: TenantDatabaseService,
|
||||||
) {}
|
) {}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Set gateway reference (called by gateway on init)
|
||||||
|
*/
|
||||||
|
setGateway(gateway: any) {
|
||||||
|
this.voiceGateway = gateway;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Get Twilio client for a tenant
|
* Get Twilio client for a tenant
|
||||||
*/
|
*/
|
||||||
@@ -487,6 +495,64 @@ export class VoiceService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Send audio data to OpenAI Realtime API
|
||||||
|
*/
|
||||||
|
async sendAudioToOpenAI(callSid: string, audioBase64: string) {
|
||||||
|
const ws = this.openaiConnections.get(callSid);
|
||||||
|
|
||||||
|
if (!ws) {
|
||||||
|
this.logger.warn(`No OpenAI connection for call ${callSid}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Send audio chunk to OpenAI
|
||||||
|
ws.send(JSON.stringify({
|
||||||
|
type: 'input_audio_buffer.append',
|
||||||
|
audio: audioBase64,
|
||||||
|
}));
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.error(`Failed to send audio to OpenAI for call ${callSid}`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Commit audio buffer to OpenAI (trigger processing)
|
||||||
|
*/
|
||||||
|
async commitAudioBuffer(callSid: string) {
|
||||||
|
const ws = this.openaiConnections.get(callSid);
|
||||||
|
|
||||||
|
if (!ws) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
ws.send(JSON.stringify({
|
||||||
|
type: 'input_audio_buffer.commit',
|
||||||
|
}));
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.error(`Failed to commit audio buffer for call ${callSid}`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Clean up OpenAI connection for a call
|
||||||
|
*/
|
||||||
|
async cleanupOpenAIConnection(callSid: string) {
|
||||||
|
const ws = this.openaiConnections.get(callSid);
|
||||||
|
|
||||||
|
if (ws) {
|
||||||
|
try {
|
||||||
|
ws.close();
|
||||||
|
this.openaiConnections.delete(callSid);
|
||||||
|
this.logger.log(`Cleaned up OpenAI connection for call ${callSid}`);
|
||||||
|
} catch (error) {
|
||||||
|
this.logger.error(`Error cleaning up OpenAI connection for call ${callSid}`, error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Handle OpenAI Realtime messages
|
* Handle OpenAI Realtime messages
|
||||||
*/
|
*/
|
||||||
@@ -505,9 +571,40 @@ export class VoiceService {
|
|||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
|
|
||||||
|
case 'response.audio.delta':
|
||||||
|
// OpenAI is sending audio response
|
||||||
|
// This needs to be sent to Twilio Media Stream
|
||||||
|
// Note: We'll need to get the streamSid from the call state
|
||||||
|
const state = this.callStates.get(callSid);
|
||||||
|
if (state?.streamSid && message.delta) {
|
||||||
|
// The controller will handle sending to Twilio
|
||||||
|
// Store audio delta for controller to pick up
|
||||||
|
if (!state.pendingAudio) {
|
||||||
|
state.pendingAudio = [];
|
||||||
|
}
|
||||||
|
state.pendingAudio.push(message.delta);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
|
||||||
|
case 'response.audio.done':
|
||||||
|
// Audio response complete
|
||||||
|
this.logger.log(`OpenAI audio response complete for call ${callSid}`);
|
||||||
|
break;
|
||||||
|
|
||||||
case 'response.audio_transcript.delta':
|
case 'response.audio_transcript.delta':
|
||||||
// Real-time transcript
|
// Real-time transcript chunk
|
||||||
// TODO: Emit to gateway
|
const deltaState = this.callStates.get(callSid);
|
||||||
|
if (deltaState?.userId && message.delta) {
|
||||||
|
// Emit to frontend via gateway
|
||||||
|
if (this.voiceGateway) {
|
||||||
|
await this.voiceGateway.notifyAiTranscript(deltaState.userId, {
|
||||||
|
callSid,
|
||||||
|
transcript: message.delta,
|
||||||
|
isFinal: false,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
this.logger.debug(`Transcript delta for call ${callSid}: ${message.delta}`);
|
||||||
|
}
|
||||||
break;
|
break;
|
||||||
|
|
||||||
case 'response.audio_transcript.done':
|
case 'response.audio_transcript.done':
|
||||||
|
|||||||
@@ -101,30 +101,50 @@ export function useSoftphone() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const { api } = useApi();
|
const { api } = useApi();
|
||||||
|
console.log('Requesting Twilio token from /api/voice/token...');
|
||||||
const response = await api.get('/voice/token');
|
const response = await api.get('/voice/token');
|
||||||
const token = response.data.token;
|
const token = response.data.token;
|
||||||
|
|
||||||
|
console.log('Token received, creating Device...');
|
||||||
|
|
||||||
|
// Log the token payload to see what identity is being used
|
||||||
|
try {
|
||||||
|
const tokenPayload = JSON.parse(atob(token.split('.')[1]));
|
||||||
|
console.log('Token identity:', tokenPayload.sub);
|
||||||
|
console.log('Token grants:', tokenPayload.grants);
|
||||||
|
} catch (e) {
|
||||||
|
console.log('Could not parse token payload');
|
||||||
|
}
|
||||||
|
|
||||||
twilioDevice.value = new Device(token, {
|
twilioDevice.value = new Device(token, {
|
||||||
logLevel: 1,
|
logLevel: 1,
|
||||||
codecPreferences: ['opus', 'pcmu'],
|
codecPreferences: ['opus', 'pcmu'],
|
||||||
enableImprovedSignalingErrorPrecision: true,
|
enableImprovedSignalingErrorPrecision: true,
|
||||||
// Specify audio constraints
|
|
||||||
edge: 'ashburn',
|
edge: 'ashburn',
|
||||||
});
|
});
|
||||||
|
|
||||||
// Device events
|
// Device events
|
||||||
twilioDevice.value.on('registered', () => {
|
twilioDevice.value.on('registered', () => {
|
||||||
console.log('Twilio Device registered');
|
console.log('✓ Twilio Device registered - ready to receive calls');
|
||||||
toast.success('Softphone ready');
|
toast.success('Softphone ready');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
twilioDevice.value.on('unregistered', () => {
|
||||||
|
console.log('⚠ Twilio Device unregistered');
|
||||||
|
});
|
||||||
|
|
||||||
twilioDevice.value.on('error', (error) => {
|
twilioDevice.value.on('error', (error) => {
|
||||||
console.error('Twilio Device error:', error);
|
console.error('❌ Twilio Device error:', error);
|
||||||
toast.error('Device error: ' + error.message);
|
toast.error('Device error: ' + error.message);
|
||||||
});
|
});
|
||||||
|
|
||||||
twilioDevice.value.on('incoming', (call: TwilioCall) => {
|
twilioDevice.value.on('incoming', (call: TwilioCall) => {
|
||||||
console.log('Incoming call:', call.parameters);
|
console.log('🔔 Twilio Device INCOMING event received:', call.parameters);
|
||||||
|
console.log('Call parameters:', {
|
||||||
|
CallSid: call.parameters.CallSid,
|
||||||
|
From: call.parameters.From,
|
||||||
|
To: call.parameters.To,
|
||||||
|
});
|
||||||
twilioCall.value = call;
|
twilioCall.value = call;
|
||||||
|
|
||||||
// Update state
|
// Update state
|
||||||
@@ -136,12 +156,27 @@ export function useSoftphone() {
|
|||||||
status: 'ringing',
|
status: 'ringing',
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// Open softphone dialog
|
||||||
|
isOpen.value = true;
|
||||||
|
|
||||||
|
// Show notification
|
||||||
|
toast.info(`Incoming call from ${incomingCall.value.fromNumber}`, {
|
||||||
|
duration: 30000,
|
||||||
|
});
|
||||||
|
|
||||||
// Setup call handlers
|
// Setup call handlers
|
||||||
setupCallHandlers(call);
|
setupCallHandlers(call);
|
||||||
|
|
||||||
|
// Play ringtone
|
||||||
|
playRingtone();
|
||||||
});
|
});
|
||||||
|
|
||||||
// Register the device
|
// Register the device
|
||||||
|
console.log('Registering Twilio Device...');
|
||||||
await twilioDevice.value.register();
|
await twilioDevice.value.register();
|
||||||
|
console.log('✓ Twilio Device register() completed');
|
||||||
|
console.log('Device identity:', twilioDevice.value.identity);
|
||||||
|
console.log('Device state:', twilioDevice.value.state);
|
||||||
|
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
console.error('Failed to initialize Twilio Device:', error);
|
console.error('Failed to initialize Twilio Device:', error);
|
||||||
@@ -320,16 +355,22 @@ export function useSoftphone() {
|
|||||||
* Accept incoming call
|
* Accept incoming call
|
||||||
*/
|
*/
|
||||||
const acceptCall = async (callSid: string) => {
|
const acceptCall = async (callSid: string) => {
|
||||||
|
console.log('📞 Accepting call - callSid:', callSid);
|
||||||
|
console.log('twilioCall.value:', twilioCall.value);
|
||||||
|
|
||||||
if (!twilioCall.value) {
|
if (!twilioCall.value) {
|
||||||
|
console.error('❌ No incoming call to accept - twilioCall.value is null');
|
||||||
toast.error('No incoming call');
|
toast.error('No incoming call');
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
|
console.log('Calling twilioCall.value.accept()...');
|
||||||
await twilioCall.value.accept();
|
await twilioCall.value.accept();
|
||||||
|
console.log('✓ Call accepted successfully');
|
||||||
toast.success('Call accepted');
|
toast.success('Call accepted');
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
console.error('Failed to accept call:', error);
|
console.error('❌ Failed to accept call:', error);
|
||||||
toast.error('Failed to accept call: ' + error.message);
|
toast.error('Failed to accept call: ' + error.message);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
@@ -397,22 +438,10 @@ export function useSoftphone() {
|
|||||||
|
|
||||||
// Event handlers
|
// Event handlers
|
||||||
const handleIncomingCall = (data: Call) => {
|
const handleIncomingCall = (data: Call) => {
|
||||||
console.log('Incoming call:', data);
|
// Socket.IO notification that a call is coming
|
||||||
incomingCall.value = data;
|
// The actual call object will come from Twilio Device SDK's 'incoming' event
|
||||||
isOpen.value = true;
|
console.log('Socket.IO call notification:', data);
|
||||||
|
// Don't set incomingCall here - wait for the Device SDK incoming event
|
||||||
toast.info(`Incoming call from ${data.fromNumber}`, {
|
|
||||||
duration: 30000,
|
|
||||||
action: {
|
|
||||||
label: 'Answer',
|
|
||||||
onClick: () => {
|
|
||||||
acceptCall(data.callSid);
|
|
||||||
},
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
// Play ringtone
|
|
||||||
playRingtone();
|
|
||||||
};
|
};
|
||||||
|
|
||||||
const handleCallInitiated = (data: any) => {
|
const handleCallInitiated = (data: any) => {
|
||||||
@@ -512,12 +541,31 @@ export function useSoftphone() {
|
|||||||
let ringtoneAudio: HTMLAudioElement | null = null;
|
let ringtoneAudio: HTMLAudioElement | null = null;
|
||||||
|
|
||||||
const playRingtone = () => {
|
const playRingtone = () => {
|
||||||
|
// Optional: Play a simple beep tone using Web Audio API
|
||||||
|
// This is a nice-to-have enhancement but not required for incoming calls to work
|
||||||
try {
|
try {
|
||||||
ringtoneAudio = new Audio('/ringtone.mp3');
|
const audioContext = new (window.AudioContext || (window as any).webkitAudioContext)();
|
||||||
ringtoneAudio.loop = true;
|
const oscillator = audioContext.createOscillator();
|
||||||
ringtoneAudio.play();
|
const gainNode = audioContext.createGain();
|
||||||
|
|
||||||
|
oscillator.connect(gainNode);
|
||||||
|
gainNode.connect(audioContext.destination);
|
||||||
|
|
||||||
|
// Phone ringtone frequency (440 Hz)
|
||||||
|
oscillator.frequency.value = 440;
|
||||||
|
oscillator.type = 'sine';
|
||||||
|
|
||||||
|
const now = audioContext.currentTime;
|
||||||
|
gainNode.gain.setValueAtTime(0.15, now);
|
||||||
|
gainNode.gain.setValueAtTime(0, now + 0.5);
|
||||||
|
gainNode.gain.setValueAtTime(0.15, now + 1.0);
|
||||||
|
gainNode.gain.setValueAtTime(0, now + 1.5);
|
||||||
|
|
||||||
|
oscillator.start(now);
|
||||||
|
oscillator.stop(now + 2);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Failed to play ringtone:', error);
|
// Silent fail - incoming call still works without audio
|
||||||
|
console.debug('Audio notification skipped:', error);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
116
validate-softphone.sh
Executable file
116
validate-softphone.sh
Executable file
@@ -0,0 +1,116 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# Softphone Incoming Call System Validation Script
|
||||||
|
# This script verifies that all components are properly configured and running
|
||||||
|
|
||||||
|
echo "╔════════════════════════════════════════════════════════════════╗"
|
||||||
|
echo "║ SOFTPHONE INCOMING CALL SYSTEM VALIDATION ║"
|
||||||
|
echo "╚════════════════════════════════════════════════════════════════╝"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Colors for output
|
||||||
|
GREEN='\033[0;32m'
|
||||||
|
RED='\033[0;31m'
|
||||||
|
YELLOW='\033[1;33m'
|
||||||
|
NC='\033[0m' # No Color
|
||||||
|
|
||||||
|
PASS=0
|
||||||
|
FAIL=0
|
||||||
|
|
||||||
|
check() {
|
||||||
|
local name=$1
|
||||||
|
local command=$2
|
||||||
|
local expected=$3
|
||||||
|
|
||||||
|
if eval "$command" > /dev/null 2>&1; then
|
||||||
|
if [ -z "$expected" ] || eval "$command" | grep -q "$expected"; then
|
||||||
|
echo -e "${GREEN}✓${NC} $name"
|
||||||
|
((PASS++))
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
echo -e "${RED}✗${NC} $name"
|
||||||
|
((FAIL++))
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "🔍 Checking Services..."
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check backend is running
|
||||||
|
check "Backend running on port 3000" "netstat -tuln | grep ':3000'" "3000"
|
||||||
|
|
||||||
|
# Check frontend is running
|
||||||
|
check "Frontend running on port 3001" "netstat -tuln | grep ':3001'" "3001"
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "🔍 Checking Backend Configuration..."
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check backend files exist
|
||||||
|
check "Voice controller exists" "test -f /root/neo/backend/src/voice/voice.controller.ts"
|
||||||
|
check "Voice gateway exists" "test -f /root/neo/backend/src/voice/voice.gateway.ts"
|
||||||
|
|
||||||
|
# Check for inbound TwiML handler
|
||||||
|
check "inboundTwiml handler defined" "grep -q '@Post.*twiml/inbound' /root/neo/backend/src/voice/voice.controller.ts"
|
||||||
|
|
||||||
|
# Check for notifyIncomingCall method
|
||||||
|
check "notifyIncomingCall method exists" "grep -q 'notifyIncomingCall' /root/neo/backend/src/voice/voice.gateway.ts"
|
||||||
|
|
||||||
|
# Check for Socket.IO emit in notifyIncomingCall
|
||||||
|
check "notifyIncomingCall emits call:incoming" "grep -A3 'notifyIncomingCall' /root/neo/backend/src/voice/voice.gateway.ts | grep -q \"call:incoming\""
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "🔍 Checking Frontend Configuration..."
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check frontend files exist
|
||||||
|
check "Softphone composable exists" "test -f /root/neo/frontend/composables/useSoftphone.ts"
|
||||||
|
check "Softphone dialog component exists" "test -f /root/neo/frontend/components/SoftphoneDialog.vue"
|
||||||
|
|
||||||
|
# Check for Socket.IO listener
|
||||||
|
check "call:incoming event listener registered" "grep -q \"'call:incoming'\" /root/neo/frontend/composables/useSoftphone.ts"
|
||||||
|
|
||||||
|
# Check for handler function
|
||||||
|
check "handleIncomingCall function defined" "grep -q 'const handleIncomingCall' /root/neo/frontend/composables/useSoftphone.ts"
|
||||||
|
|
||||||
|
# Check that handler updates incomingCall ref
|
||||||
|
check "Handler updates incomingCall.value" "grep -A5 'const handleIncomingCall' /root/neo/frontend/composables/useSoftphone.ts | grep -q 'incomingCall.value = data'"
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "🔍 Checking End-to-End Flow..."
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Check that backend calls notifyIncomingCall in handler
|
||||||
|
check "inboundTwiml calls notifyIncomingCall" "grep -A50 '@Post.*twiml/inbound' /root/neo/backend/src/voice/voice.controller.ts | grep -q 'notifyIncomingCall'"
|
||||||
|
|
||||||
|
# Check TwiML generation includes Dial
|
||||||
|
check "TwiML includes Dial element" "grep -A50 '@Post.*twiml/inbound' /root/neo/backend/src/voice/voice.controller.ts | grep -q '<Dial'"
|
||||||
|
|
||||||
|
# Check TwiML includes Client elements
|
||||||
|
check "TwiML includes Client dial targets" "grep -A50 '@Post.*twiml/inbound' /root/neo/backend/src/voice/voice.controller.ts | grep -q '<Client>'"
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
echo "╔════════════════════════════════════════════════════════════════╗"
|
||||||
|
echo "║ VALIDATION SUMMARY ║"
|
||||||
|
echo "╠════════════════════════════════════════════════════════════════╣"
|
||||||
|
printf "║ %-50s %s ║\n" "Tests Passed" "${GREEN}${PASS}${NC}"
|
||||||
|
printf "║ %-50s %s ║\n" "Tests Failed" "${RED}${FAIL}${NC}"
|
||||||
|
echo "╚════════════════════════════════════════════════════════════════╝"
|
||||||
|
|
||||||
|
if [ $FAIL -eq 0 ]; then
|
||||||
|
echo ""
|
||||||
|
echo -e "${GREEN}✓ All checks passed! System is properly configured.${NC}"
|
||||||
|
echo ""
|
||||||
|
echo "Next Steps:"
|
||||||
|
echo "1. Connect to softphone at http://localhost:3001"
|
||||||
|
echo "2. Open softphone dialog and verify it shows 'Connected' status"
|
||||||
|
echo "3. Make an inbound call to your Twilio number"
|
||||||
|
echo "4. Verify incoming call dialog appears in softphone UI"
|
||||||
|
echo "5. Test accepting/rejecting the call"
|
||||||
|
exit 0
|
||||||
|
else
|
||||||
|
echo ""
|
||||||
|
echo -e "${RED}✗ Some checks failed. Review the configuration.${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
Reference in New Issue
Block a user