JavaScript SDK Overview

Build web applications with Pipecat’s JavaScript client library

The Pipecat JavaScript SDK provides a lightweight client implementation that handles:

• Device and media stream management
• Connecting to Pipecat bots
• Messaging with Pipecat bots and handling responses using the RTVI standard
• Managing session state and errors

Installation

Install the SDK and a transport implementation (e.g. Daily for WebRTC):

npm install @pipecat-ai/client-js
npm install @pipecat-ai/[daily-transport, small-webrtc-transport, etc.]

Example

Here’s a simple example using Daily as the transport layer:

import { PipecatClient } from "@pipecat-ai/client-js";
import { DailyTransport } from "@pipecat-ai/daily-transport";

// Handle incoming audio from the bot
function handleBotAudio(track, participant) {
  if (participant.local || track.kind !== "audio") return;

  const audioElement = document.createElement("audio");
  audioElement.srcObject = new MediaStream([track]);
  document.body.appendChild(audioElement);
  audioElement.play();
}

// Create and configure the client
const pcClient = new PipecatClient({
  transport: new DailyTransport(),
  enableMic: true,
  callbacks: {
    onTrackStarted: handleBotAudio,
  },
});

// Connect to your bot
pcClient.connect({
  url: "https://your-daily-room-url",
  token: "your-daily-token",
});

Explore the SDK

Client Constructor

Configure your client instance with transport and callbacks

Client Methods

Core methods for interacting with your bot

Callbacks & Events

Handle bot events, messages, and state changes

Transport Packages

Daily, SmallWebRTC, WebSocket, and other transports

UI Agent Protocol

Send UI events and dispatch UI commands with PipecatClient

A11y Snapshots

Stream the document’s accessibility tree to the server as <ui_state>

The Pipecat JavaScript SDK implements the RTVI standard for real-time AI inference, ensuring compatibility with any RTVI-compatible server and transport layer.