How to Add a Real-Time Talking Avatar to an ElevenLabs Agent, Step by Step
- Published:
- Reading time:
- 10 min
To give an ElevenLabs agent a real-time face, run its conversation through your own server. Open ElevenLabs' Agents WebSocket there, pass each audio event to a Talking Avatar Realistic Avatars session with pushAudio(), call clear() on each interruption, and show the avatar on your page with LiveAvatarView. Set the agent's output to PCM at 24 kHz first. The avatar costs $0.02 a minute, billed per second, on top of your ElevenLabs usage.
Key takeaways
- ElevenLabs' own Avatars make talking-head videos from a script. A live agent needs an avatar rendered in real time from its voice.
- The integration is one Node.js file of about 100 lines: ElevenLabs' Agents WebSocket on one side, a Talking Avatar Realistic Avatars session on the other.
- Two settings in ElevenLabs make it work: TTS output as PCM 24000 Hz, and user input as PCM (16000 Hz is plenty for speech).
- The avatar starts speaking less than 250 ms after the agent's audio reaches it and costs $0.02 a minute. New accounts get $1 of free credits, about 50 minutes.
ElevenLabs Avatars aren't live: what an agent needs instead
In June 2026, ElevenLabs added Avatars to ElevenCreative: you choose an avatar, write a script, pick a voice and generate a talking-head video. That suits course videos and ads, but it can't hold a conversation. An ElevenLabs agent answers in real time, so its face has to be rendered in real time too, from the agent's voice as it streams.
ElevenLabs documents one real-time avatar integration for its agents, HeyGen LiveAvatar, and other providers publish their own guides (we compare them in the best real-time AI avatar APIs). This guide uses Talking Avatar Realistic Avatars: photorealistic avatars rendered on our servers and streamed to your page over WebRTC, for $0.02 a minute.
How the integration works
- The visitor clicks Talk. Your server starts an avatar session, and the page starts playing it.
- Once the avatar is live, your server opens a conversation on ElevenLabs' Agents WebSocket with a signed URL, so your ElevenLabs API key never leaves the server.
- The page streams the microphone to your server, which forwards it to ElevenLabs as
user_audio_chunkmessages. - Each
audioevent from ElevenLabs goes toavatar.pushAudio(). The avatar speaks it with lip sync and plays the voice in the page. - When ElevenLabs hears the visitor talk over the agent, it sends
interruption, andavatar.clear()stops the avatar mid-sentence.
The browser never talks to ElevenLabs directly. Avatar sessions run on your server, so the agent's voice has to pass through it on the way to the avatar.
What you need
- An ElevenLabs agent and an ElevenLabs API key.
- A Talking Avatar account. Sign up to get $1 of free credits; no card is needed.
- Node.js 22 or later on your server.
- A current Chrome, Edge, Firefox or Safari for the page.
Step 1: Set the agent's audio to PCM
The avatar plays raw 16-bit PCM at 8, 12, 16, 24 or 48 kHz. In your agent's settings in ElevenLabs, set:
- Voice → TTS output format: PCM 24000 Hz.
- Advanced → User input audio format: PCM 16000 Hz. Any PCM rate works, because the page converts the microphone to it.
ElevenLabs' own avatar guide changes the same two settings and sets both to 24000 Hz, which works here too. Outputs at 22,050 or 44,100 Hz won't play on the avatar; if the agent sends one, the server below stops and tells you which setting to change.
Step 2: Get your keys
- ElevenLabs API key: Settings → API keys.
- ElevenLabs agent ID: copy it from your agent in Agents.
- Talking Avatar API key: in the dashboard, open API keys → Create key.
Put them in a .env file on your server. Neither key ever reaches the browser.
TALKING_AVATAR_API_KEY=tav_…
ELEVENLABS_API_KEY=…
ELEVENLABS_AGENT_ID=…Step 3: Install the packages
Step 4: Write the server
This file serves the page, starts an avatar for each visitor and relays audio between the visitor, the ElevenLabs agent and the avatar.
import { randomUUID } from "node:crypto";
import express from "express";
import { WebSocket, WebSocketServer } from "ws";
import { LiveAvatar } from "@talkingavatar/sdk/server";
const { TALKING_AVATAR_API_KEY, ELEVENLABS_API_KEY, ELEVENLABS_AGENT_ID } = process.env;
const RATES = [8000, 12000, 16000, 24000, 48000]; // the sample rates the avatar accepts
const avatars = new Map(); // conversation ID → LiveAvatar
const app = express()
.use(express.json())
.use(express.static("public"))
.use("/sdk", express.static("node_modules/@talkingavatar/sdk/dist")); // the browser SDK, no bundler needed
// The page's WebRTC route: plays one conversation's avatar.
app.post("/avatar/:id", async (req, res) => {
const avatar = avatars.get(req.params.id);
if (!avatar) return res.status(404).json({ error: "No such conversation" });
try { res.json(await avatar.handleView(req.body)); }
catch (error) { res.status(503).json({ error: error.message }); }
});
const server = app.listen(3000, (error) => {
if (!error) console.log("Open http://localhost:3000"); // on an error, such as the port in use, Node prints it and exits
});
// One WebSocket per visitor: their microphone in, the conversation's state out.
new WebSocketServer({ server, path: "/conversation" }).on("connection", async (browser) => {
const id = randomUUID();
let avatar, agent, ended = false;
const end = (reason) => {
if (ended) return;
ended = true;
agent?.close();
avatar?.close(); // billing stops
avatars.delete(id);
if (browser.readyState === WebSocket.OPEN) {
browser.send(JSON.stringify({ type: "ended", reason }));
browser.close();
}
};
browser.on("close", () => end("The visitor left"));
// 1. Start the avatar.
try {
avatar = await LiveAvatar.start({ apiKey: TALKING_AVATAR_API_KEY, avatar: "ava4", sampleRate: 24000 });
} catch (error) {
return end("The avatar didn't start: " + error.message);
}
if (ended) return avatar.close(); // the visitor left while it started
avatars.set(id, avatar);
avatar.on("close", (code) => end("The avatar session ended (" + code + ")")); // 4002: out of credits
browser.send(JSON.stringify({ type: "session", id }));
browser.on("message", async (data, isBinary) => {
// 4. The visitor's microphone, already 16-bit PCM at the agent's input rate.
if (isBinary) {
if (agent?.readyState === WebSocket.OPEN) agent.send(JSON.stringify({ user_audio_chunk: data.toString("base64") }));
return;
}
// 2. The visitor can see and hear the avatar: start the conversation now, so they hear the first message.
let message;
try { message = JSON.parse(data.toString()); }
catch { return end("Invalid browser message"); }
if (!message || typeof message !== "object" || Array.isArray(message) || typeof message.type !== "string") return end("Invalid browser message");
if (message.type !== "ready" || agent) return;
try {
agent = await connectAgent();
} catch (error) {
return end(error.message);
}
if (ended) return agent.close(); // the visitor left while it connected
let outputRate = 24000;
agent.on("message", (raw) => {
const event = JSON.parse(raw.toString());
switch (event.type) {
case "conversation_initiation_metadata": {
const { agent_output_audio_format: output, user_input_audio_format: input } = event.conversation_initiation_metadata_event;
outputRate = Number(output.replace("pcm_", ""));
if (!output.startsWith("pcm_") || !RATES.includes(outputRate)) return end("Set the agent's TTS output format to PCM 24000 Hz (it's " + output + ")");
if (!input.startsWith("pcm_")) return end("Set the agent's user input audio format to PCM 16000 Hz (it's " + input + ")");
browser.send(JSON.stringify({ type: "listen", sampleRate: Number(input.replace("pcm_", "")) }));
break;
}
// 3. The agent's voice: the avatar speaks it with lip sync.
case "audio":
avatar.pushAudio(event.audio_event.audio_base_64, { sampleRate: outputRate });
break;
// 5. The visitor talked over the agent: stop the avatar mid-sentence.
case "interruption":
avatar.clear().catch(() => {});
break;
case "ping":
agent.send(JSON.stringify({ type: "pong", event_id: event.ping_event.event_id }));
break;
}
});
agent.on("close", () => end("The conversation ended"));
agent.on("error", (error) => end("ElevenLabs: " + error.message));
});
});
// A signed URL keeps your ElevenLabs API key on the server.
async function connectAgent() {
const url = "https://api.elevenlabs.io/v1/convai/conversation/get-signed-url?agent_id=" + ELEVENLABS_AGENT_ID;
const response = await fetch(url, { headers: { "xi-api-key": ELEVENLABS_API_KEY } });
if (!response.ok) throw new Error("ElevenLabs refused the signed URL (" + response.status + ")");
const { signed_url } = await response.json();
const agent = new WebSocket(signed_url);
await new Promise((resolve, reject) => { agent.once("open", resolve); agent.once("error", reject); });
agent.send(JSON.stringify({ type: "conversation_initiation_client_data" }));
return agent;
}
What the numbered parts do:
- 1. Starts one avatar session per visitor. It's billed per second until
close(). - 2. Waits for the page to report the avatar as live before it opens the ElevenLabs conversation, so the agent's first message isn't spoken to an empty page.
- 3. Passes each
audioevent's base64 PCM straight topushAudio(). ElevenLabs can send audio faster than real time; the avatar queues it and speaks it at its natural pace. - 4. Forwards the microphone to ElevenLabs as
user_audio_chunk. - 5. On
interruption,clear()drops whatever the avatar hasn't said yet. - It also answers ElevenLabs'
pingwithpong, checks the agent's audio formats when the conversation starts, and ends everything when either side closes.
ava4 is one of our library avatars; swap in any avatar ID, or your own custom avatar's.
Step 5: Stream the microphone
Browsers record at 44.1 or 48 kHz in floating point. This AudioWorklet converts the microphone to 16-bit PCM at the agent's input rate and sends it in 100 ms chunks.
// Resamples the microphone to the agent's input rate and posts 16-bit PCM in 100 ms chunks.
class Mic extends AudioWorkletProcessor {
constructor({ processorOptions }) {
super();
this.step = sampleRate / processorOptions.rate; // input samples per output sample
this.at = 0; // read position; -1 is the previous block's last sample
this.last = 0;
this.chunk = new Int16Array(processorOptions.rate / 10);
this.length = 0;
}
process([input]) {
const samples = input[0];
if (!samples) return true;
while (this.at < samples.length - 1) {
const i = Math.floor(this.at);
const a = i < 0 ? this.last : samples[i];
const value = a + (samples[i + 1] - a) * (this.at - i);
this.chunk[this.length++] = Math.max(-1, Math.min(1, value)) * 0x7fff;
if (this.length === this.chunk.length) {
const out = this.chunk.slice();
this.port.postMessage(out.buffer, [out.buffer]);
this.length = 0;
}
this.at += this.step;
}
this.at -= samples.length;
this.last = samples[samples.length - 1];
return true;
}
}
registerProcessor("mic", Mic);
Step 6: Show the avatar on your page
LiveAvatarView plays the avatar's video and the agent's voice, in sync, and reconnects by itself. The server above serves the SDK's browser file at /sdk/live.js, so this page needs no bundler; in an app with one, import it from @talkingavatar/sdk/live.
<!doctype html>
<meta charset="utf-8">
<title>ElevenLabs agent with a Talking Avatar</title>
<div id="avatar" style="width: 640px; max-width: 100%; aspect-ratio: 16 / 9; background: #111"></div>
<p>
<button id="start">Talk to the agent</button>
<button id="unmute" hidden>Turn sound on</button>
<span id="status"></span>
</p>
<script type="module" src="/page.js"></script>
import { LiveAvatarView } from "/sdk/live.js";
const start = document.getElementById("start");
const unmute = document.getElementById("unmute");
const status = document.getElementById("status");
start.onclick = async () => {
start.disabled = true;
// Echo cancellation keeps the avatar's own voice out of the microphone.
let mic;
try {
mic = await navigator.mediaDevices.getUserMedia({ audio: { echoCancellation: true, noiseSuppression: true } });
} catch (error) {
status.textContent = "Microphone unavailable: " + error.message;
start.disabled = false;
return;
}
const socket = new WebSocket(location.origin.replace("http", "ws") + "/conversation");
let view, audio, ready = false;
socket.onmessage = async ({ data }) => {
const message = JSON.parse(data);
if (message.type === "session") {
// Plays the avatar's video and the agent's voice.
view = new LiveAvatarView({
container: document.getElementById("avatar"),
endpoint: "/avatar/" + message.id,
onStatus: (s) => {
status.textContent = s;
unmute.hidden = s !== "muted";
if ((s === "live" || s === "muted") && !ready) {
ready = true;
socket.send(JSON.stringify({ type: "ready" }));
}
},
});
unmute.onclick = () => view.unmute();
view.connect();
}
if (message.type === "listen") {
audio = await streamMicrophone(mic, message.sampleRate, (pcm) => {
if (socket.readyState === WebSocket.OPEN) socket.send(pcm);
});
}
if (message.type === "ended") {
view?.disconnect(); // first, so its "closed" status doesn't replace the reason
status.textContent = message.reason;
}
};
socket.onclose = () => {
view?.disconnect();
view?.video.remove();
audio?.close();
mic.getTracks().forEach((track) => track.stop());
start.disabled = false;
};
};
// Sends the microphone as 16-bit PCM at the agent's input rate, in 100 ms chunks.
async function streamMicrophone(stream, rate, send) {
const context = new AudioContext();
await context.audioWorklet.addModule("/mic-worklet.js");
const node = new AudioWorkletNode(context, "mic", { processorOptions: { rate } });
node.port.onmessage = ({ data }) => send(data);
context.createMediaStreamSource(stream).connect(node).connect(context.destination); // the node outputs silence
await context.resume();
return context;
}
- Play the voice once. The avatar plays the agent's voice itself. Don't also play ElevenLabs' audio anywhere else on the page.
- Sound blocked? If the browser blocks sound, the status is
mutedand the page shows a Turn sound on button. - Echo cancellation keeps the avatar's voice out of the microphone, so the agent doesn't hear itself.
Step 7: Run it
node --env-file=.env server.jsOpen http://localhost:3000, click Talk to the agent and allow the microphone. The avatar says the agent's first message, then listens. Talk over it to check that it stops mid-sentence.
Before you go live
- Protect
/conversation. Every connection starts a billed avatar session, so put it behind your app's login and limit how many each user can open. - Serve the page over HTTPS. Browsers only allow the microphone on secure pages (localhost is the exception).
- Add a TURN server through
LiveAvatarView'siceServersoption, for visitors behind strict firewalls. - Set a monthly spending limit and auto top-up in the dashboard. A session ends with close code 4002 when credits run out.
- Plan for the limits. One avatar session lasts up to 3 hours. A small “Powered by Talking Avatar” watermark shows unless you remove it for $19 a month.
What it costs
Talking Avatar Realistic Avatars cost $0.02 a minute, billed per second, with no subscription; high-volume contracts go as low as $0.005 a minute. ElevenLabs bills the agent separately, from $0.08/min, plus LLM on its pricing page (checked 4 November 2026).
| At 1,000 minutes a month | Cost |
|---|---|
| Avatar (Talking Avatar Realistic Avatars) | $20 |
| Agent (ElevenLabs) | from $80, plus the LLM |
Don't need ElevenLabs specifically? Talking Avatar Voice Agents are complete voice agents on a realistic avatar, set up in the dashboard without code, for $0.06 a minute with the avatar included at no extra cost.
Troubleshooting
- “Set the agent’s TTS output format to PCM 24000 Hz”
- The agent sends a format the avatar can't play. Change it as in step 1.
- “ElevenLabs refused the signed URL (401 or 404)”
- 401: check
ELEVENLABS_API_KEY, and that the key is allowed to use Agents. 404: checkELEVENLABS_AGENT_ID. - “The avatar didn’t start: talking-avatar: invalid API key”
- Check
TALKING_AVATAR_API_KEY, or create a new key in the dashboard. - The avatar moves but makes no sound
- The browser blocked sound (status
muted). Click Turn sound on, which callsview.unmute(). - You hear the agent twice
- Something else also plays ElevenLabs' audio. Remove it: the avatar plays the voice itself.
- The agent keeps interrupting itself
- Its voice is reaching the microphone. Keep
echoCancellationon, and test with headphones.
Methodology
How we wrote this guide
We build Talking Avatar Realistic Avatars, so the avatar side follows our SDK documentation for @talkingavatar/sdk 0.1.0. The ElevenLabs side follows ElevenLabs' Agents WebSocket documentation and its avatar guide's audio settings, read on 9 November 2026. ElevenLabs' agent price comes from its pricing page, checked on 4 November 2026. The avatar latency figure is our own measurement in production.
Frequently asked questions
Does ElevenLabs have a real-time avatar?
ElevenLabs doesn't document a real-time avatar of its own. ElevenLabs Avatars, in ElevenCreative, generate talking-head videos from a script. For live agents, ElevenLabs documents an integration with HeyGen LiveAvatar, and other avatar providers, Talking Avatar among them, connect through ElevenLabs' Agents WebSocket.
How do I add an avatar to an ElevenLabs agent?
Set the agent's TTS output format to PCM 24000 Hz. On your server, open ElevenLabs' Agents WebSocket with a signed URL, pass each audio event to a Talking Avatar Realistic Avatars session with pushAudio(), and call clear() on each interruption event. Show the avatar on your page with LiveAvatarView, and stream the microphone to your server.
Can the ElevenLabs conversation stay in the browser?
Not with Talking Avatar Realistic Avatars: avatar sessions run on your server, so the agent's voice has to reach it. Run the conversation over ElevenLabs' Agents WebSocket from your server, as this guide does.
Does it work with ElevenLabs text-to-speech, without an agent?
Yes. Ask ElevenLabs for output_format pcm_24000, start the avatar with sampleRate 24000, and pass the audio to avatar.pushAudio() as it streams. Our Realistic Avatars quickstart shows the rest.
What does an avatar on an ElevenLabs agent cost?
Talking Avatar Realistic Avatars cost $0.02 a minute, billed per second, with no subscription. ElevenLabs bills the agent separately, from $0.08/min, plus LLM on its published pricing. At 1,000 minutes a month, that's $20 for the avatar and from $80 for ElevenLabs.
Can the avatar look like me?
Yes. Pick any avatar from our library, or order a custom avatar of a real person, with their consent, for $199 one time. Use its ID in LiveAvatar.start().

About me
Mukul Munjal
Co-founder, Talking Avatar
I'm Mukul Munjal, co-founder of Talking Avatar. I have 7 years of experience in engineering, 3 of them building voice agents, and I handle the optimisation of our voice agents. We built Talking Avatar to give our own AI interviewers a face.
Evaluating tools?
Compare your options
See how the platforms in this article compare on pricing, features and fit.
HeyGen LiveAvatar
Anam
D-ID Agents
Realistic avatars from $0.02 a minute, billed per second. No subscription.
Avatar API prices comparedGet API key