2026 – present · Planswell, now FUTR
AI Caller
I built a real-time AI calling system that talks with prospective financial-planning clients, and it now makes calls every day. The challenge was making it responsive and natural while handling the realities of phone calls: interruptions, background noise, voicemail, ambiguous answers and unreliable outside services.
It runs on a streaming voice pipeline: an Elixir service handles call control, and a Python worker combines speech recognition, language models and speech synthesis over live Twilio audio. Speculative response preparation, reusable audio and compact conversation memory keep waits short. Interruption handling tells speech apart from noise and speakerphone echo, lowers playback while it checks, and stops talking at a word boundary.
Guardrails respect contact preferences, prevent duplicate calls and won’t record a booking unless the conversation actually supports it. Simulated conversations test the dialogue without placing a call.