What is an AI voice agent?
An AI voice agent is software that holds a conversation on your behalf, on the phone or in a browser. It listens, works out what the person wants, answers from your own knowledge, takes actions in your systems such as booking or order lookup, and transfers to a person when needed. A custom one runs on infrastructure you control.
It suits any business whose calls follow a handful of patterns at volume: service companies booking jobs, retailers answering order questions, lenders sending reminders, software companies qualifying leads. Owning the stack is about cost and control: production minutes on a custom LiveKit stack come to about 2.5 cents, against about 10 cents on a managed platform, and every speech, model and voice vendor stays replaceable.
Most teams start with one agent on one number and add agents for other call types on the same platform. That is the version priced here: inbound and outbound calling, a knowledge base, tools into one of your systems, transfers, and a dashboard for agent settings, call logs and analytics. Reselling it to clients as a white-label platform is an add-on further down.
Can interrupt, change their mind and ask a follow-up, and gets the task done in one call without a phone menu.
Changes prompts, knowledge and transfer rules in the dashboard, tests them with a call, and sees why each call ended as it did.
Owns every component in its own cloud account, pays each vendor at list price and can replace any of them without a rebuild.
What features does an AI voice agent need?
An AI voice agent needs 8 core features: inbound and outbound, versioned agent settings, knowledge with sources, tools that take action, transfers with context, natural turn-taking, every call replayable and vendors you can swap.
Inbound and outbound
Answers your numbers and dials out from a list, a schedule or a CRM event, with voicemail detection on every outbound call.
Versioned agent settings
Prompt, greeting, voice, language and turn-taking live in versioned configs you can test with a call and roll back.
Knowledge with sources
Policies, FAQs and product pages are indexed, and every answer is logged with the passage it came from.
Tools that take action
Order lookups, bookings, CRM updates and ticket creation run as typed tool calls with timeouts, and changes are read back before they are made.
Transfers with context
Hands the call to a number or a queue with a short summary for the person picking up, and books a callback if nobody answers.
Natural turn-taking
Stops talking the moment the caller interrupts and waits through a thinking pause instead of jumping in.
Every call replayable
Recording, transcript, tool calls, latency per turn and cost for each call, searchable by outcome and intent.
Vendors you can swap
Speech, model and voice providers sit behind one interface, so trying a cheaper or better one is a configuration change.
What screens does an AI voice agent have?
It is built around 3 screens: agent editor, live call and calls and analytics.
- 1Agent editorThe agent's greeting, voice, knowledge sources, enabled tools and outbound campaign, with a test call button.
- 2Live callThe customer's phone during a call, with the agent checking a repair order and answering in one turn.
- 3Calls and analyticsThe week's calls, resolution and transfer rates, average cost per call and recent calls by intent.
How does an AI voice agent work?
End to end, in 5 steps: the call connects, speech becomes text, the agent answers or acts, people take over when needed and each call becomes data.
- 1
The call connects
Inbound calls reach LiveKit through your SIP trunk, and outbound calls are created by the dialer from a list or a CRM event. Each call gets its own worker process.
- 2
Speech becomes text
Streaming speech-to-text transcribes while the person talks, and a turn detector decides when they have finished rather than just paused.
- 3
The agent answers or acts
The model replies from retrieved knowledge or calls a tool, such as looking up an order or booking a slot, and confirms any change before making it.
- 4
People take over when needed
Anything outside the agent's rules is handed to your team along with a short summary, or saved as a callback task when nobody can take it.
- 5
Each call becomes data
Recording, transcript, tool calls, latency and cost are stored per call and rolled up into outcomes by intent on the dashboard.
What is the architecture and tech stack of an AI voice agent?
It has 8 layers: telephony (Twilio or Telnyx SIP trunks with LiveKit SIP), real-time voice (LiveKit server and LiveKit Agents), speech to text (Deepgram Nova-3 or Flux, streaming), language model (Claude Haiku 4.5 on live turns, Claude Sonnet 5 after the call), text to speech (Cartesia or ElevenLabs), knowledge and records (Postgres with pgvector, recordings in S3), dashboard (Next.js on Vercel) and agent hosting (Agent workers on AWS, autoscaled by load). The diagram shows how a request moves through them.
| Layer | What we use | Why |
|---|---|---|
| Telephony | Twilio or Telnyx SIP trunks with LiveKit SIP | Keep your numbers; the same trunk carries inbound calls and the dialer's outbound ones. |
| Real-time voice | LiveKit server and LiveKit Agents | Built for live audio: it dispatches a worker to every call, handles barge-in, and can run entirely in your own cloud. |
| Speech to text | Deepgram Nova-3 or Flux, streaming | Partial transcripts arrive while the caller is still talking, and keyterm prompting fixes product and street names. |
| Language model | Claude Haiku 4.5 on live turns, Claude Sonnet 5 after the call | The small model keeps replies inside the latency budget; the larger one scores and summarizes calls when speed no longer matters. |
| Text to speech | Cartesia or ElevenLabs | Fast first audio, and a choice of voices you can match to your brand. |
| Knowledge and records | Postgres with pgvector, recordings in S3 | Knowledge, agent configs and call records in one database you own, and recordings in your own bucket. |
| Dashboard | Next.js on Vercel | Agent settings, call logs and analytics for the people who run the phones, not only the engineers. |
| Agent hosting | Agent workers on AWS, autoscaled by load | More workers start as calls rise, and old ones finish their calls before they stop, so releasing an update never cuts a caller off. |
How much does it cost to build an AI voice agent?
A launch-ready AI voice agent costs $25,500 to $50,500 to build and takes 7 to 11 weeks. A clickable demo costs $3,100 to $6,000 (2 to 4 weeks), and running it costs $310 to $560 a month at the usage below. You start at $0 and pay per checkpoint you accept.
Priced with the same model as our AI product cost estimator, from the features above. Your price is fixed in writing after a 20-minute call, before any work starts.
| Version | Build cost | Timeline | What it is |
|---|---|---|---|
| Clickable demo | $3,100 to $6,000 | 2 to 4 weeks | Clickable and real where it matters, on test data. Built to show users and investors, not to carry production traffic, so compliance work starts at launch. |
| Launch-ready | $25,500 to $50,500 | 7 to 11 weeks | Production architecture, tests on the risky paths, monitoring, and a handover your team can run. |
| Enterprise-grade | $33,500 to $66,000 | 8 to 14 weeks | Load tested, highly available, audited and documented for a larger team. |
What it costs to run
Around 10,000 minutes of inbound and outbound calls each month (about 110 three-minute calls a day) and a small team using the dashboard.
| Line | Per month | Assumes |
|---|---|---|
| Hosting and database | $45 to $120 | Vercel + managed services, sized for 100 monthly users |
| Model usage | $15 to $35 | Claude Sonnet 5, 15 requests per user a month |
| Voice minutes | $250 | 10,000 minutes on Custom LiveKit stack |
| Email, monitoring, analytics | $0 to $150 | Free tiers cover most products at launch |
| Total | $310 to $560 | List prices, before any volume discount |
Build at $0: how you pay
$0 is when you pay, not what you pay. The launch-ready build is split into checkpoints with acceptance criteria agreed before work starts, and each one is invoiced only after you have seen it and accepted it.
- 1Scope and acceptance criteriaBefore work startsA call, then a written plan: every checkpoint with acceptance criteria you agree to before work starts.$0
- 2Architecture and first flowBy week 2Data model, service boundaries and one real flow working end to end.$5,000 to $10,000
- 3Core productBy week 6The main flows on production architecture, with a demo at the end of every week.$7,500 to $15,000
- 4AI on your real dataBy week 9Models, agents or voice working on real inputs, with evals and guardrails in place.$7,500 to $15,000
- 5Launch and handoverBy week 11Deployed on your accounts and documented, with 30 days of defect correction included.$5,000 to $10,000
What can you add to an AI voice agent after launch?
The additions most teams make next: white-label for your clients, voice on your website and app, test agent versions on live calls and callers in their own language.
White-label for your clients
Sub-accounts per client, per-minute usage billing and your own brand on the dashboard, for agencies and software companies reselling voice.
Voice on your website and app
The same agent inside your site or mobile app over WebRTC, sharing its knowledge, tools and call logs with the phone line.
Test agent versions on live calls
Split traffic between two agent versions and compare resolution, transfers and handle time before promoting one.
Callers in their own language
Detects the caller's language in the first turn and switches speech model, voice and knowledge to match.
What are the risks when building an AI voice agent?
Three things decide whether it works in production: reply time is the product, outbound calls carry TCPA rules and build only past the break-even.
Reply time is the product
Callers start talking over an agent that takes longer than about 800 milliseconds to answer. Stream every stage, keep workers in the same region as your telephony, and alert on the 95th percentile turn time, not the average.
Outbound calls carry TCPA rules
The FCC ruled in February 2024 that AI-generated voices are artificial voices under the TCPA, so calls to mobiles need prior express consent, and written consent for marketing. Check consent, calling hours and do-not-call lists in code before the dialer fires.
Build only past the break-even
On our arithmetic a managed platform is cheaper below roughly 20,000 minutes a month once engineering and on-call time are counted. Above it, paying each vendor at list price wins, provided someone owns incidents and vendor changes.
Where can you read more before you build?
- How to Build Your Own Voice AI Platform on LiveKit: Architecture, Costs and a Build Plan
- Retell vs Vapi vs Bland vs Custom LiveKit (2026): Price per Minute, Latency and Lock-In
- Voice AI Latency: How to Get a Voice Agent Under 800ms (Full Budget Breakdown)
- AI Voice Agent Development: the service behind this build
