Voice-Enabled Personal Agent
A hybrid voice agent that receives inbound calls, queues tasks for background processing, and makes outbound calls to notify you when tasks complete or need input.
Features
- Inbound Call Handling: Answer calls, transcribe voice commands, extract tasks
- Task Queue: Background processing with retry logic
- Outbound Callbacks: Notify via call or SMS when tasks complete
- Multi-Executor Support: Email, Calendar, Browser automation, Code execution
- Claude Code Integration: Delegate coding tasks to Claude Code subprocess
- Git Commit Tracking: Auto-detect commits, send WhatsApp notifications with diffs
- WhatsApp Integration: Real-time notifications via Twilio or 360dialog
- Local-First: Works offline with SQLite, or scales with Supabase
Architecture
┌─────────────────────────────────────────────────────────────────┐
│ YOUR MACHINE │
│ ┌────────────────────────────────────────────────────────────┐ │
│ │ VOICE AGENT SYSTEM │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌──────────────┐ │ │
│ │ │ Inbound Call│────▶│ Task Queue │───▶│ Task Worker │ │ │
│ │ │ (Telnyx) │ │ (Supabase) │ │ (Background) │ │ │
│ │ └─────────────┘ └─────────────┘ └──────┬───────┘ │ │
│ │ │ │ │ │
│ │ ▼ ▼ │ │
│ │ ┌─────────────┐ ┌──────────────┐ │ │
│ │ │ Claude Agent│◀──────────────────────│ Executors │ │ │
│ │ │ (Brain) │ │ Email/Cal/ │ │ │
│ │ └──────┬──────┘ │ Browser/Code │ │ │
│ │ │ └──────────────┘ │ │
│ │ ▼ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ │ │ │
│ │ │Outbound Call│◀────│ Callback │◀─────────┘ │ │
│ │ │ (Telnyx) │ │ Queue │ │ │
│ │ └─────────────┘ └─────────────┘ │ │
│ └────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
Sequence Diagrams
Open the .puml files in VS Code with PlantUML extension (Alt+D to preview):
Quick Start
1. Install Dependencies
bun install
2. Configure Environment
cp .env.example .env
# Edit .env with your API keys
3. Run Setup
bun run setup
4. Start Development Server
# Start main server
bun run dev
# In another terminal, start task worker
bun run dev:worker
5. Configure Google OAuth (optional)
Visit http://localhost:3333/oauth/google to connect your Google account for email and calendar features.
Configuration
Required Environment Variables
| Variable | Description |
|---|
CALLME_PHONE_ACCOUNT_SID | Telnyx Connection ID |
CALLME_PHONE_AUTH_TOKEN | Telnyx API Key |
CALLME_PHONE_NUMBER | Your Telnyx phone number |
CALLME_USER_PHONE_NUMBER | Phone number to call you at |
CALLME_OPENAI_API_KEY | OpenAI API key (for STT/TTS) |
ANTHROPIC_API_KEY | Claude API key |
CALLME_NGROK_AUTHTOKEN | ngrok auth token |
Optional Environment Variables
| Variable | Description |
|---|
SUPABASE_URL | Supabase project URL |
SUPABASE_SERVICE_KEY | Supabase service key |
GOOGLE_CLIENT_ID | Google OAuth client ID |
GOOGLE_CLIENT_SECRET | Google OAuth secret |
POSTHOG_API_KEY | PostHog analytics key |
USE_LOCAL_STORAGE | Set to true for SQLite |
TWILIO_ACCOUNT_SID | Twilio SID for WhatsApp |
TWILIO_AUTH_TOKEN | Twilio auth token |
TWILIO_WHATSAPP_FROM | Twilio WhatsApp sender number |
Task Types
Email
Send and read emails via Gmail.
{
type: 'email',
params: {
to: 'recipient@example.com',
subject: 'Meeting Tomorrow',
body: 'Hi, let\'s meet at 3pm.'
}
}
Calendar
Manage Google Calendar events.