Talking Avatar Open Source · Developer guide · JavaScript and React

Open-source talking avatar SDK

Add a talking 3D character to your app with @interviewflowai/talking-avatar. The SDK's 1 MB model turns audio into mouth shapes and animates your .glb avatar in real time, in the user's browser. The code and lip-sync model are MIT licensed, free for commercial and noncommercial use, with no API key or avatar usage fees.

View the GitHub repository Try the 3D browser demo

Install the open-source talking avatar npm package

The package uses ES modules and includes TypeScript declarations. Install three alongside it for 3D rendering (version 0.160.0 or later). React 18 or later is needed only for the React component. The same package also contains the clients for Talking Avatar Realistic Avatars, the separate hosted product: its /server and /live imports need an API key and are billed per minute. The root and /react imports on this page are the free open-source SDK.

npm install @interviewflowai/talking-avatar three

Prefer another package manager? Use pnpm add @interviewflowai/talking-avatar three or yarn add @interviewflowai/talking-avatar three. See the package on npm →

Prepare your avatar, audio and model

Put a .glb avatar with the 15 Oculus viseme morph targets in your public assets as /avatar.glb. Use morphTargets if the names differ.
Add a recording or TTS audio file as /hello.mp3. The SDK animates audio you supply; speech generation is separate.
Create public/sdk and copy the packaged model there with the command below. Both examples self-host it.
Give the avatar container a width and height, then start audio from a click or tap.

cp node_modules/@interviewflowai/talking-avatar/dist/viseme-model.bin public/sdk/viseme-model.bin

Alternatively, omit modelUrl to load this package version's model from jsDelivr. Host avatar and audio assets on the same origin, or configure CORS on the asset host.

JavaScript and React quickstart

Choose the class for a plain JavaScript app, or the component for React. Replace the example asset paths with your own files. React handles disposal on unmount; with the class, call dispose() yourself.

import { TalkingAvatar } from "@interviewflowai/talking-avatar";

// Add a sized <div id="avatar"> and a <button id="play">.
const avatar = new TalkingAvatar({
  container: document.getElementById("avatar"),
  avatarUrl: "/avatar.glb",
  modelUrl: "/sdk/viseme-model.bin",
  onError: console.error,
});

document.getElementById("play").onclick = () => {
  avatar.speak("/hello.mp3").catch(console.error);
};

// Call avatar.dispose() when you remove this view.

"use client"; // Needed in Next.js App Router.
import { useRef } from "react";
import { TalkingAvatarView } from "@interviewflowai/talking-avatar/react";

export function Avatar() {
  const avatar = useRef(null);
  return (
    <>
      <div style={{ width: "100%", height: 360 }}>
        <TalkingAvatarView
          ref={avatar}
          avatarUrl="/avatar.glb"
          modelUrl="/sdk/viseme-model.bin"
          onError={console.error}
        />
      </div>
      <button onClick={() => {
        avatar.current?.speak("/hello.mp3").catch(console.error);
      }}>Play audio</button>
    </>
  );
}

For live audio, pass a MediaStream to attachStream() or the React stream prop. See the open-source talking avatar API reference for constructor options, stream handling, playback and cleanup.

Voice agents and talking mascots

Use your voice agent's remote audio as the input, whether it comes from OpenAI Realtime, LiveKit, Daily or another WebRTC integration. A tutor, interviewer or support assistant can speak through the same 3D avatar. A brand mascot works too if its .glb model has compatible mouth shapes; the SDK does not create or rig mascot artwork.

Free to use and modify

The MIT License permits commercial and noncommercial use, modification and redistribution. Keep its copyright and permission notice with copies or substantial portions of the software. Avatar assets, TTS and hosting have their own terms.

See what the free talking avatar includes for a demo and answers about costs, compatible avatars and free use.

Browser requirements and best results

Use a browser with WebGL and Web Audio support. Live audio uses AudioWorklet and needs HTTPS or localhost. Start audio after a user interaction, and let the avatar play its own audio to avoid echo. Clean TTS or recorded speech gives more reliable lip sync than noisy microphones. The model was trained on English speech, so mouth shapes for other languages are approximate.

The GitHub README documents avatar requirements, model training, troubleshooting and contributions. You can inspect the implementation, report an issue or contribute a fix directly in the repository.