callme

Voice-Enabled Personal Agent

A hybrid voice agent that receives inbound calls, queues tasks for background processing, and makes outbound calls to notify you when tasks complete or need input.

Features

• Inbound Call Handling: Answer calls, transcribe voice commands, extract tasks
• Task Queue: Background processing with retry logic
• Outbound Callbacks: Notify via call or SMS when tasks complete
• Multi-Executor Support: Email, Calendar, Browser automation, Code execution
• Claude Code Integration: Delegate coding tasks to Claude Code subprocess
• Git Commit Tracking: Auto-detect commits, send WhatsApp notifications with diffs
• WhatsApp Integration: Real-time notifications via Twilio or 360dialog
• Local-First: Works offline with SQLite, or scales with Supabase

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     YOUR MACHINE                                 │
│  ┌────────────────────────────────────────────────────────────┐ │
│  │              VOICE AGENT SYSTEM                             │ │
│  │                                                             │ │
│  │  ┌─────────────┐     ┌─────────────┐    ┌──────────────┐  │ │
│  │  │ Inbound Call│────▶│ Task Queue  │───▶│ Task Worker  │  │ │
│  │  │ (Telnyx)    │     │ (Supabase)  │    │ (Background) │  │ │
│  │  └─────────────┘     └─────────────┘    └──────┬───────┘  │ │
│  │        │                                        │          │ │
│  │        ▼                                        ▼          │ │
│  │  ┌─────────────┐                       ┌──────────────┐   │ │
│  │  │ Claude Agent│◀──────────────────────│ Executors    │   │ │
│  │  │ (Brain)     │                       │ Email/Cal/   │   │ │
│  │  └──────┬──────┘                       │ Browser/Code │   │ │
│  │         │                              └──────────────┘   │ │
│  │         ▼                                      │          │ │
│  │  ┌─────────────┐     ┌─────────────┐          │          │ │
│  │  │Outbound Call│◀────│ Callback    │◀─────────┘          │ │
│  │  │ (Telnyx)    │     │ Queue       │                     │ │
│  │  └─────────────┘     └─────────────┘                     │ │
│  └────────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

Sequence Diagrams

Open the .puml files in VS Code with PlantUML extension (Alt+D to preview):

Diagram	File
WhatsApp → Code → Commit	docs/diagrams/1-whatsapp-code-execution.puml
Task Queue Flow	docs/diagrams/2-task-queue-flow.puml
Memory & Context	docs/diagrams/3-memory-flow.puml
Voice Call Flow	docs/diagrams/4-voice-call-flow.puml
System Overview	docs/diagrams/5-system-overview.puml

Quick Start

1. Install Dependencies

bun install

2. Configure Environment

cp .env.example .env
# Edit .env with your API keys

3. Run Setup

bun run setup

4. Start Development Server

# Start main server
bun run dev

# In another terminal, start task worker
bun run dev:worker

5. Configure Google OAuth (optional)

Visit http://localhost:3333/oauth/google to connect your Google account for email and calendar features.

Configuration

Required Environment Variables

Variable	Description
CALLME_PHONE_ACCOUNT_SID	Telnyx Connection ID
CALLME_PHONE_AUTH_TOKEN	Telnyx API Key
CALLME_PHONE_NUMBER	Your Telnyx phone number
CALLME_USER_PHONE_NUMBER	Phone number to call you at
CALLME_OPENAI_API_KEY	OpenAI API key (for STT/TTS)
ANTHROPIC_API_KEY	Claude API key
CALLME_NGROK_AUTHTOKEN	ngrok auth token

Optional Environment Variables

Variable	Description
SUPABASE_URL	Supabase project URL
SUPABASE_SERVICE_KEY	Supabase service key
GOOGLE_CLIENT_ID	Google OAuth client ID
GOOGLE_CLIENT_SECRET	Google OAuth secret
POSTHOG_API_KEY	PostHog analytics key
USE_LOCAL_STORAGE	Set to true for SQLite
TWILIO_ACCOUNT_SID	Twilio SID for WhatsApp
TWILIO_AUTH_TOKEN	Twilio auth token
TWILIO_WHATSAPP_FROM	Twilio WhatsApp sender number

Task Types

Email

Send and read emails via Gmail.

{
  type: 'email',
  params: {
    to: 'recipient@example.com',
    subject: 'Meeting Tomorrow',
    body: 'Hi, let\'s meet at 3pm.'
  }
}

Calendar

Manage Google Calendar events.