Talking Avatar
← Back to BlogIntegration guide

How to Add a Real-Time Talking Avatar to an ElevenLabs Agent, Step by Step

Published:
Reading time:
10 min
Illustration of connecting a voice agent to a live talking avatar: a card with a sound-wave badge and a microphone, a server card with a code symbol, and a browser window where a realistic avatar speaks, linked by a flowing line

To give an ElevenLabs agent a real-time face, run its conversation through your own server. Open ElevenLabs' Agents WebSocket there, pass each audio event to a Talking Avatar Realistic Avatars session with pushAudio(), call clear() on each interruption, and show the avatar on your page with LiveAvatarView. Set the agent's output to PCM at 24 kHz first. The avatar costs $0.02 a minute, billed per second, on top of your ElevenLabs usage.

Key takeaways

  • ElevenLabs' own Avatars make talking-head videos from a script. A live agent needs an avatar rendered in real time from its voice.
  • The integration is one Node.js file of about 100 lines: ElevenLabs' Agents WebSocket on one side, a Talking Avatar Realistic Avatars session on the other.
  • Two settings in ElevenLabs make it work: TTS output as PCM 24000 Hz, and user input as PCM (16000 Hz is plenty for speech).
  • The avatar starts speaking less than 250 ms after the agent's audio reaches it and costs $0.02 a minute. New accounts get $1 of free credits, about 50 minutes.

ElevenLabs Avatars aren't live: what an agent needs instead

In June 2026, ElevenLabs added Avatars to ElevenCreative: you choose an avatar, write a script, pick a voice and generate a talking-head video. That suits course videos and ads, but it can't hold a conversation. An ElevenLabs agent answers in real time, so its face has to be rendered in real time too, from the agent's voice as it streams.

ElevenLabs documents one real-time avatar integration for its agents, HeyGen LiveAvatar, and other providers publish their own guides (we compare them in the best real-time AI avatar APIs). This guide uses Talking Avatar Realistic Avatars: photorealistic avatars rendered on our servers and streamed to your page over WebRTC, for $0.02 a minute.

How the integration works

How an ElevenLabs agent gets a real-time avatarThe browser sends the microphone to your server, which relays audio both ways with the ElevenLabs agent over a WebSocket and passes the agent's voice to Talking Avatar. Talking Avatar streams the avatar's video and voice back to the browser over WebRTC.Visitor's browsermicrophone + LiveAvatarViewYour serverNode.js bridgeElevenLabs agentlistens, thinks, speaksTalking Avatarrenders the facemicrophoneaudio, WebSocketpushAudio(), clear()video and voice, WebRTC
  1. The visitor clicks Talk. Your server starts an avatar session, and the page starts playing it.
  2. Once the avatar is live, your server opens a conversation on ElevenLabs' Agents WebSocket with a signed URL, so your ElevenLabs API key never leaves the server.
  3. The page streams the microphone to your server, which forwards it to ElevenLabs as user_audio_chunk messages.
  4. Each audio event from ElevenLabs goes to avatar.pushAudio(). The avatar speaks it with lip sync and plays the voice in the page.
  5. When ElevenLabs hears the visitor talk over the agent, it sends interruption, and avatar.clear() stops the avatar mid-sentence.

The browser never talks to ElevenLabs directly. Avatar sessions run on your server, so the agent's voice has to pass through it on the way to the avatar.

What you need

  • An ElevenLabs agent and an ElevenLabs API key.
  • A Talking Avatar account. Sign up to get $1 of free credits; no card is needed.
  • Node.js 22 or later on your server.
  • A current Chrome, Edge, Firefox or Safari for the page.

Step 1: Set the agent's audio to PCM

The avatar plays raw 16-bit PCM at 8, 12, 16, 24 or 48 kHz. In your agent's settings in ElevenLabs, set:

  • Voice → TTS output format: PCM 24000 Hz.
  • Advanced → User input audio format: PCM 16000 Hz. Any PCM rate works, because the page converts the microphone to it.

ElevenLabs' own avatar guide changes the same two settings and sets both to 24000 Hz, which works here too. Outputs at 22,050 or 44,100 Hz won't play on the avatar; if the agent sends one, the server below stops and tells you which setting to change.

Step 2: Get your keys

Put them in a .env file on your server. Neither key ever reaches the browser.

.env
TALKING_AVATAR_API_KEY=tav_…
ELEVENLABS_API_KEY=…
ELEVENLABS_AGENT_ID=…

Step 3: Install the packages

In a new folder, install our SDK, Express for the web server and ws for the WebSockets:

npm init -y && npm pkg set type=module && npm install @talkingavatar/sdk express ws

Step 4: Write the server

This file serves the page, starts an avatar for each visitor and relays audio between the visitor, the ElevenLabs agent and the avatar.

server.js
import { randomUUID } from "node:crypto";
import express from "express";
import { WebSocket, WebSocketServer } from "ws";
import { LiveAvatar } from "@talkingavatar/sdk/server";

const { TALKING_AVATAR_API_KEY, ELEVENLABS_API_KEY, ELEVENLABS_AGENT_ID } = process.env;
const RATES = [8000, 12000, 16000, 24000, 48000]; // the sample rates the avatar accepts
const avatars = new Map();                        // conversation ID → LiveAvatar

const app = express()
  .use(express.json())
  .use(express.static("public"))
  .use("/sdk", express.static("node_modules/@talkingavatar/sdk/dist")); // the browser SDK, no bundler needed

// The page's WebRTC route: plays one conversation's avatar.
app.post("/avatar/:id", async (req, res) => {
  const avatar = avatars.get(req.params.id);
  if (!avatar) return res.status(404).json({ error: "No such conversation" });
  try { res.json(await avatar.handleView(req.body)); }
  catch (error) { res.status(503).json({ error: error.message }); }
});

const server = app.listen(3000, (error) => {
  if (!error) console.log("Open http://localhost:3000"); // on an error, such as the port in use, Node prints it and exits
});

// One WebSocket per visitor: their microphone in, the conversation's state out.
new WebSocketServer({ server, path: "/conversation" }).on("connection", async (browser) => {
  const id = randomUUID();
  let avatar, agent, ended = false;

  const end = (reason) => {
    if (ended) return;
    ended = true;
    agent?.close();
    avatar?.close(); // billing stops
    avatars.delete(id);
    if (browser.readyState === WebSocket.OPEN) {
      browser.send(JSON.stringify({ type: "ended", reason }));
      browser.close();
    }
  };
  browser.on("close", () => end("The visitor left"));

  // 1. Start the avatar.
  try {
    avatar = await LiveAvatar.start({ apiKey: TALKING_AVATAR_API_KEY, avatar: "ava4", sampleRate: 24000 });
  } catch (error) {
    return end("The avatar didn't start: " + error.message);
  }
  if (ended) return avatar.close(); // the visitor left while it started
  avatars.set(id, avatar);
  avatar.on("close", (code) => end("The avatar session ended (" + code + ")")); // 4002: out of credits
  browser.send(JSON.stringify({ type: "session", id }));

  browser.on("message", async (data, isBinary) => {
    // 4. The visitor's microphone, already 16-bit PCM at the agent's input rate.
    if (isBinary) {
      if (agent?.readyState === WebSocket.OPEN) agent.send(JSON.stringify({ user_audio_chunk: data.toString("base64") }));
      return;
    }
    // 2. The visitor can see and hear the avatar: start the conversation now, so they hear the first message.
    let message;
    try { message = JSON.parse(data.toString()); }
    catch { return end("Invalid browser message"); }
    if (!message || typeof message !== "object" || Array.isArray(message) || typeof message.type !== "string") return end("Invalid browser message");
    if (message.type !== "ready" || agent) return;
    try {
      agent = await connectAgent();
    } catch (error) {
      return end(error.message);
    }
    if (ended) return agent.close(); // the visitor left while it connected
    let outputRate = 24000;

    agent.on("message", (raw) => {
      const event = JSON.parse(raw.toString());
      switch (event.type) {
        case "conversation_initiation_metadata": {
          const { agent_output_audio_format: output, user_input_audio_format: input } = event.conversation_initiation_metadata_event;
          outputRate = Number(output.replace("pcm_", ""));
          if (!output.startsWith("pcm_") || !RATES.includes(outputRate)) return end("Set the agent's TTS output format to PCM 24000 Hz (it's " + output + ")");
          if (!input.startsWith("pcm_")) return end("Set the agent's user input audio format to PCM 16000 Hz (it's " + input + ")");
          browser.send(JSON.stringify({ type: "listen", sampleRate: Number(input.replace("pcm_", "")) }));
          break;
        }
        // 3. The agent's voice: the avatar speaks it with lip sync.
        case "audio":
          avatar.pushAudio(event.audio_event.audio_base_64, { sampleRate: outputRate });
          break;
        // 5. The visitor talked over the agent: stop the avatar mid-sentence.
        case "interruption":
          avatar.clear().catch(() => {});
          break;
        case "ping":
          agent.send(JSON.stringify({ type: "pong", event_id: event.ping_event.event_id }));
          break;
      }
    });
    agent.on("close", () => end("The conversation ended"));
    agent.on("error", (error) => end("ElevenLabs: " + error.message));
  });
});

// A signed URL keeps your ElevenLabs API key on the server.
async function connectAgent() {
  const url = "https://api.elevenlabs.io/v1/convai/conversation/get-signed-url?agent_id=" + ELEVENLABS_AGENT_ID;
  const response = await fetch(url, { headers: { "xi-api-key": ELEVENLABS_API_KEY } });
  if (!response.ok) throw new Error("ElevenLabs refused the signed URL (" + response.status + ")");
  const { signed_url } = await response.json();
  const agent = new WebSocket(signed_url);
  await new Promise((resolve, reject) => { agent.once("open", resolve); agent.once("error", reject); });
  agent.send(JSON.stringify({ type: "conversation_initiation_client_data" }));
  return agent;
}

What the numbered parts do:

  • 1. Starts one avatar session per visitor. It's billed per second until close().
  • 2. Waits for the page to report the avatar as live before it opens the ElevenLabs conversation, so the agent's first message isn't spoken to an empty page.
  • 3. Passes each audio event's base64 PCM straight to pushAudio(). ElevenLabs can send audio faster than real time; the avatar queues it and speaks it at its natural pace.
  • 4. Forwards the microphone to ElevenLabs as user_audio_chunk.
  • 5. On interruption, clear() drops whatever the avatar hasn't said yet.
  • It also answers ElevenLabs' ping with pong, checks the agent's audio formats when the conversation starts, and ends everything when either side closes.

ava4 is one of our library avatars; swap in any avatar ID, or your own custom avatar's.

Step 5: Stream the microphone

Browsers record at 44.1 or 48 kHz in floating point. This AudioWorklet converts the microphone to 16-bit PCM at the agent's input rate and sends it in 100 ms chunks.

public/mic-worklet.js
// Resamples the microphone to the agent's input rate and posts 16-bit PCM in 100 ms chunks.
class Mic extends AudioWorkletProcessor {
  constructor({ processorOptions }) {
    super();
    this.step = sampleRate / processorOptions.rate; // input samples per output sample
    this.at = 0;                                     // read position; -1 is the previous block's last sample
    this.last = 0;
    this.chunk = new Int16Array(processorOptions.rate / 10);
    this.length = 0;
  }

  process([input]) {
    const samples = input[0];
    if (!samples) return true;
    while (this.at < samples.length - 1) {
      const i = Math.floor(this.at);
      const a = i < 0 ? this.last : samples[i];
      const value = a + (samples[i + 1] - a) * (this.at - i);
      this.chunk[this.length++] = Math.max(-1, Math.min(1, value)) * 0x7fff;
      if (this.length === this.chunk.length) {
        const out = this.chunk.slice();
        this.port.postMessage(out.buffer, [out.buffer]);
        this.length = 0;
      }
      this.at += this.step;
    }
    this.at -= samples.length;
    this.last = samples[samples.length - 1];
    return true;
  }
}

registerProcessor("mic", Mic);

Step 6: Show the avatar on your page

LiveAvatarView plays the avatar's video and the agent's voice, in sync, and reconnects by itself. The server above serves the SDK's browser file at /sdk/live.js, so this page needs no bundler; in an app with one, import it from @talkingavatar/sdk/live.

public/index.html
<!doctype html>
<meta charset="utf-8">
<title>ElevenLabs agent with a Talking Avatar</title>
<div id="avatar" style="width: 640px; max-width: 100%; aspect-ratio: 16 / 9; background: #111"></div>
<p>
  <button id="start">Talk to the agent</button>
  <button id="unmute" hidden>Turn sound on</button>
  <span id="status"></span>
</p>
<script type="module" src="/page.js"></script>
public/page.js
import { LiveAvatarView } from "/sdk/live.js";

const start = document.getElementById("start");
const unmute = document.getElementById("unmute");
const status = document.getElementById("status");

start.onclick = async () => {
  start.disabled = true;
  // Echo cancellation keeps the avatar's own voice out of the microphone.
  let mic;
  try {
    mic = await navigator.mediaDevices.getUserMedia({ audio: { echoCancellation: true, noiseSuppression: true } });
  } catch (error) {
    status.textContent = "Microphone unavailable: " + error.message;
    start.disabled = false;
    return;
  }
  const socket = new WebSocket(location.origin.replace("http", "ws") + "/conversation");
  let view, audio, ready = false;

  socket.onmessage = async ({ data }) => {
    const message = JSON.parse(data);
    if (message.type === "session") {
      // Plays the avatar's video and the agent's voice.
      view = new LiveAvatarView({
        container: document.getElementById("avatar"),
        endpoint: "/avatar/" + message.id,
        onStatus: (s) => {
          status.textContent = s;
          unmute.hidden = s !== "muted";
          if ((s === "live" || s === "muted") && !ready) {
            ready = true;
            socket.send(JSON.stringify({ type: "ready" }));
          }
        },
      });
      unmute.onclick = () => view.unmute();
      view.connect();
    }
    if (message.type === "listen") {
      audio = await streamMicrophone(mic, message.sampleRate, (pcm) => {
        if (socket.readyState === WebSocket.OPEN) socket.send(pcm);
      });
    }
    if (message.type === "ended") {
      view?.disconnect(); // first, so its "closed" status doesn't replace the reason
      status.textContent = message.reason;
    }
  };

  socket.onclose = () => {
    view?.disconnect();
    view?.video.remove();
    audio?.close();
    mic.getTracks().forEach((track) => track.stop());
    start.disabled = false;
  };
};

// Sends the microphone as 16-bit PCM at the agent's input rate, in 100 ms chunks.
async function streamMicrophone(stream, rate, send) {
  const context = new AudioContext();
  await context.audioWorklet.addModule("/mic-worklet.js");
  const node = new AudioWorkletNode(context, "mic", { processorOptions: { rate } });
  node.port.onmessage = ({ data }) => send(data);
  context.createMediaStreamSource(stream).connect(node).connect(context.destination); // the node outputs silence
  await context.resume();
  return context;
}
  • Play the voice once. The avatar plays the agent's voice itself. Don't also play ElevenLabs' audio anywhere else on the page.
  • Sound blocked? If the browser blocks sound, the status is muted and the page shows a Turn sound on button.
  • Echo cancellation keeps the avatar's voice out of the microphone, so the agent doesn't hear itself.

Step 7: Run it

node --env-file=.env server.js

Open http://localhost:3000, click Talk to the agent and allow the microphone. The avatar says the agent's first message, then listens. Talk over it to check that it stops mid-sentence.

Before you go live

  • Protect /conversation. Every connection starts a billed avatar session, so put it behind your app's login and limit how many each user can open.
  • Serve the page over HTTPS. Browsers only allow the microphone on secure pages (localhost is the exception).
  • Add a TURN server through LiveAvatarView's iceServers option, for visitors behind strict firewalls.
  • Set a monthly spending limit and auto top-up in the dashboard. A session ends with close code 4002 when credits run out.
  • Plan for the limits. One avatar session lasts up to 3 hours. A small “Powered by Talking Avatar” watermark shows unless you remove it for $19 a month.

Read the full production checklist →

What it costs

Talking Avatar Realistic Avatars cost $0.02 a minute, billed per second, with no subscription; high-volume contracts go as low as $0.005 a minute. ElevenLabs bills the agent separately, from $0.08/min, plus LLM on its pricing page (checked 4 November 2026).

Monthly cost of an ElevenLabs agent with a realistic avatar at 1,000 minutes
At 1,000 minutes a monthCost
Avatar (Talking Avatar Realistic Avatars)$20
Agent (ElevenLabs)from $80, plus the LLM

Don't need ElevenLabs specifically? Talking Avatar Voice Agents are complete voice agents on a realistic avatar, set up in the dashboard without code, for $0.06 a minute with the avatar included at no extra cost.

Troubleshooting

“Set the agent’s TTS output format to PCM 24000 Hz”
The agent sends a format the avatar can't play. Change it as in step 1.
“ElevenLabs refused the signed URL (401 or 404)”
401: check ELEVENLABS_API_KEY, and that the key is allowed to use Agents. 404: check ELEVENLABS_AGENT_ID.
“The avatar didn’t start: talking-avatar: invalid API key”
Check TALKING_AVATAR_API_KEY, or create a new key in the dashboard.
The avatar moves but makes no sound
The browser blocked sound (status muted). Click Turn sound on, which calls view.unmute().
You hear the agent twice
Something else also plays ElevenLabs' audio. Remove it: the avatar plays the voice itself.
The agent keeps interrupting itself
Its voice is reaching the microphone. Keep echoCancellation on, and test with headphones.

Methodology

How we wrote this guide

We build Talking Avatar Realistic Avatars, so the avatar side follows our SDK documentation for @talkingavatar/sdk 0.1.0. The ElevenLabs side follows ElevenLabs' Agents WebSocket documentation and its avatar guide's audio settings, read on 9 November 2026. ElevenLabs' agent price comes from its pricing page, checked on 4 November 2026. The avatar latency figure is our own measurement in production.

Frequently asked questions

Does ElevenLabs have a real-time avatar?

ElevenLabs doesn't document a real-time avatar of its own. ElevenLabs Avatars, in ElevenCreative, generate talking-head videos from a script. For live agents, ElevenLabs documents an integration with HeyGen LiveAvatar, and other avatar providers, Talking Avatar among them, connect through ElevenLabs' Agents WebSocket.

How do I add an avatar to an ElevenLabs agent?

Set the agent's TTS output format to PCM 24000 Hz. On your server, open ElevenLabs' Agents WebSocket with a signed URL, pass each audio event to a Talking Avatar Realistic Avatars session with pushAudio(), and call clear() on each interruption event. Show the avatar on your page with LiveAvatarView, and stream the microphone to your server.

Can the ElevenLabs conversation stay in the browser?

Not with Talking Avatar Realistic Avatars: avatar sessions run on your server, so the agent's voice has to reach it. Run the conversation over ElevenLabs' Agents WebSocket from your server, as this guide does.

Does it work with ElevenLabs text-to-speech, without an agent?

Yes. Ask ElevenLabs for output_format pcm_24000, start the avatar with sampleRate 24000, and pass the audio to avatar.pushAudio() as it streams. Our Realistic Avatars quickstart shows the rest.

What does an avatar on an ElevenLabs agent cost?

Talking Avatar Realistic Avatars cost $0.02 a minute, billed per second, with no subscription. ElevenLabs bills the agent separately, from $0.08/min, plus LLM on its published pricing. At 1,000 minutes a month, that's $20 for the avatar and from $80 for ElevenLabs.

Can the avatar look like me?

Yes. Pick any avatar from our library, or order a custom avatar of a real person, with their consent, for $199 one time. Use its ID in LiveAvatar.start().

Mukul Munjal

About me

Mukul Munjal

Co-founder, Talking Avatar

I'm Mukul Munjal, co-founder of Talking Avatar. I have 7 years of experience in engineering, 3 of them building voice agents, and I handle the optimisation of our voice agents. We built Talking Avatar to give our own AI interviewers a face.

Connect with me on LinkedInThe team behind Talking Avatar

Evaluating tools?

Compare your options

See how the platforms in this article compare on pricing, features and fit.

Realistic avatars from $0.02 a minute, billed per second. No subscription.

Avatar API prices comparedGet API key