UI Prompts

Canonical prompt fragments describing the UI agent wire format

Overview

Apps concatenate these constants into their system prompt so the LLM understands the <ui_state> and <ui_event> developer messages the SDK injects on its behalf.

from pipecat_subagents.agents import UI_STATE_PROMPT_GUIDE

The SDK updates the guide alongside the wire format, so apps get new tags and semantics automatically on the next release.

UI_STATE_PROMPT_GUIDE

UI_STATE_PROMPT_GUIDE: str

A multi-line string that documents:

• The two SDK-managed message kinds (<ui_event> and <ui_state>).
• The Playwright-MCP-style indented format and what each line means.
• Known state tags ([focused], [selected], [disabled], [offscreen]).
• Grid metadata ([cols=N]) and how to compute row/column from the flat reading order.
• How to resolve position references (“top right”, “the third one”) against the most recent <ui_state> tree.
• The <selection ref="eN">selected text</selection> block that appears inside <ui_state> when the user has text selected on the page, and how to treat it as the deictic referent for “this” / “that” / “what I selected.”

Use it directly inside an f-string when building your system prompt:

from pipecat_subagents.agents import UI_STATE_PROMPT_GUIDE

system_prompt = f"""\
You are a voice-driven music player agent.

...app-specific tool and behavior instructions...

{UI_STATE_PROMPT_GUIDE}
"""