如需通过 Gemini Live API 获得更好的结果,请重点关注以下最佳实践:
设计清晰的系统指令
为了充分发挥 Gemini Live API 的性能,建议您有一套清晰定义的系统指令 (SI),按照智能体角色、对话规则和保护措施的顺序进行定义。
为获得最佳结果,请将每个智能体分离到不同的 SI 中。
指定智能体的角色设定:详细说明智能体的名称、角色和任何偏好特征。如果您想指定口音,请务必同时指定首选输出语言(例如,针对说英语的人指定英式口音)。
指定对话规则:按您希望模型遵循的顺序放置这些规则。划分对话中一次性元素和对话循环之间的界限。例如:
- 一次性元素:一次性收集客户的详细信息(例如姓名、位置、会员卡号)。
- 对话循环:用户可以讨论建议、价格、退货和配送,并可能希望在不同主题之间切换。让模型知道,只要用户愿意,就可以一直进行这种对话循环。
在流程中以单独的句子指定工具调用:例如,如果收集客户详细信息的一次性步骤需要调用一个
get_user_info函数,您可以这样说:第一步是收集用户信息。首先,要求用户提供其姓名、位置信息和会员卡号。然后,使用这些详细信息调用get_user_info。添加任何必要的保护措施:提供您不希望模型执行的任何通用对话保护措施。您可以随意提供具体示例,例如,说明如果发生 x,您希望模型执行 y。如果您仍未获得所需的精确度,请使用“毫无疑问”一词来引导模型提高精确度。
精确定义工具
将工具与 Gemini Live API 搭配使用时,工具定义应具体明确。务必告知 Gemini 在什么条件下应调用工具。如需了解详情,请参阅工具定义。
撰写有效的提示
使用清晰的提示:在提示中提供模型应该和不应该做什么的示例,并尽量限制每个提示只针对一个角色或身份。考虑使用提示链代替冗长的多页提示。模型在涉及单一函数调用的任务上表现最佳。
# Prompt chaining example. chainable_long_prompt = """ You need to perform a sequence of tasks. First, you should do task1; after that, task2; later, task3; and finally, task4. """ # New initial prompt """ You need to perform a sequence of tasks. Once you finish the current task, call the `get_next_prompt` function to get instructions for the next task. """ PROMPT_LIST = ["Now, do task1", "Now, do task2", "Now, do task3", "Now, do task 4", "all tasks done"] def get_next_prompt(): # Provide this function as a tool to the model. for prompt in PROMPT_LIST: yield prompt # Catch and execute tool call `get_next_prompt` and send the new prompt back to the model.提供起始命令和信息:Gemini Live API 在响应前需要用户输入。如要让 Gemini Live API 发起对话,请添加一个要求它向用户问好或开始对话的提示。添加有关用户的信息,以便 Gemini Live API 对问候语进行个性化设置。
会话恢复
- 使用透明会话恢复:在
genai.types.LiveConnectConfig中使用SessionResumptionConfig(transparent=True)配置连接。这表示客户端打算无缝处理会话恢复,从而实现重新连接时重放未使用的消息等功能。
from google.genai import types
session_handle: str | None = None
live_config = types.LiveConnectConfig(
session_resumption=types.SessionResumptionConfig(
handle=session_handle,
transparent=True,
),
)
维护和更新会话句柄:监听来自服务器的
session_resumption_update消息。如果resumable为 true 且提供了new_handle,则存储此句柄。如果发生断开连接,此句柄对于重新连接到同一会话状态至关重要。缓冲已发送的消息并剪除已确认的消息:为确保在断开连接期间不会丢失任何客户端消息,请维护一个发送给 Gemini Live API 的消息缓冲区。启用透明会话恢复时,
session_resumption_update消息将包含last_consumed_client_message_index,用于指示服务器处理的最后一条消息。使用此索引从缓冲区中移除已确认的消息。为正确跟踪消息,用户管理的索引必须从 1 开始,因为索引 0 表示the session is not resumable。发送给模型的每条后续消息都应将此索引递增 1。每次会话恢复时,请确保使用新连接传输的初始消息的索引重置为 1。从容处理断开连接的情况:
- GoAway 信号:服务器在预期断开连接(例如超时)之前发送
go_away消息。管理器应监听此消息,然后使用最新句柄主动重新连接。 - API 错误:网络问题可能会导致
genai_errors.APIError(例如,WebSocket 错误的代码 1000 或 1006)。管理器应在发送和接收循环中捕获这些错误,并触发会话更新或重新连接过程。
- GoAway 信号:服务器在预期断开连接(例如超时)之前发送
实现消息重放的重新连接:当发生断开连接时,使用
client.aio.live.connect和最新的会话句柄创建新会话。建立新连接后,重新发送缓冲区中在断开连接之前未被服务器确认的任何消息。缓冲区中发送的第一条消息应标记为新连接的索引 1。
启用上下文窗口压缩
使用 ContextWindowCompressionConfig 配置长会话的上下文窗口,因为原生音频令牌会快速累积(大约每秒音频 25 个令牌)。
警告:上下文压缩会导致对话记录丢失。
from google.genai import types
live_config = types.LiveConnectConfig(
context_window_compression=types.ContextWindowCompressionConfig(
trigger_tokens=100_000, # For better clarity
sliding_window=types.SlidingWindow(target_tokens=4_000),
),
)
token 用量计算
价格页面上详细介绍了 Gemini Live API 的结算结构。
在每个回合中,API 会针对所有上下文令牌收费,其中包括对话历史记录和用户提供的系统指令。开发者可以通过提取模型响应中提供的 usage_metadata 字段来监控和计算这些费用。
# Example code to get token usage
from google.genai import live
session: live.AsyncSession
async for response in session.receive():
if response.usage_metadata is not None:
print("Token usage:", response.usage_metadata)
语音活动检测 (VAD)
默认情况下,Gemini Live API 使用 Gemini 提供的 VAD。
使用 Gemini Live API VAD 时,您可以将模型配置为明确返回 VAD 事件。通过在配置中启用 explicit_vad_signal,您可以直接从模型的回答中监控和捕获这些事件。
from google.genai import types
from google.genai import live
live_config = types.LiveConnectConfig(
explicit_vad_signal=True
)
session: live.AsyncSession
# In receive loop
async for response in session.receive():
if response.voice_activity is not None:
print("Get VAD event", response.voice_activity)
如果您想使用自定义活动检测系统,则必须停用默认的语音活动检测 (VAD),并手动向 Gemini 模型发送用户轮流发言信号。为此,您需要传输 ActivityStart 或 ActivityEnd 事件来定义互动边界。
from google.genai import live
from google.genai import types
# Disable VAD in config
live_config = types.LiveConnectConfig(
realtime_input_config=types.RealtimeInputConfig(
automatic_activity_detection=types.AutomaticActivityDetection(
disabled=True
),
),
)
session: live.AsyncSession
await session.send_realtime_input( # Send activity start
activity_start=types.ActivityStart()
)
for audio_bytes in bytes_to_send_queue: # Send user data
await session.send_realtime_input(
audio=types.Blob(
data=audio_bytes,
mime_type=f"audio/pcm;rate=16000",
)
)
await session.send_realtime_input(activity_end=types.ActivityEnd()) # Send activity end
设置音频语言代码
建议在配置中明确设置语言和语音代码,以保持一致性;如果没有此定义,Gemini 可能会根据提供的上下文更改对话语言。
from google.genai import types
config = types.LiveConnectConfig(
speech_config=types.SpeechConfig(
language_code="en-US",
),
)
此外,还可以在系统指令中提及以下内容:
RESPOND IN {OUTPUT_LANGUAGE}. YOU MUST RESPOND UNMISTAKABLY IN {OUTPUT_LANGUAGE}.
对于 gemini-live-2.5-flash-native-audio 等原生音频模型,您可以在会话配置中提供语言提示,以提高多语言自动语音识别 (ASR) 的转写质量。如需了解详情,请参阅为会话启用音频转写。
设置转写语言代码
使用 BCP-47 语言代码格式指定转写语言代码,以提高转写准确度。
注意:启用转写功能会增加令牌用量。
from google.genai import types
config = types.LiveConnectConfig(
input_audio_transcription=types.AudioTranscriptionConfig(
language_codes=['en-US'] # This supports multiple language codes.
),
output_audio_transcription=types.AudioTranscriptionConfig(
language_codes=['en-US']
),
)
客户端缓冲
在发送之前,不要大幅缓冲输入音频(例如 1 秒)。发送小块(20 毫秒到 40 毫秒之间)以最大限度地减少延迟时间。
重新采样
确保您的客户端应用在传输之前将麦克风输入(通常为 44.1 kHz 或 48 kHz)重新采样为 16 kHz。
示例
此示例结合了最佳实践和系统指令设计指南,旨在引导模型发挥职业规划师的作用。
**Persona:**
You are Laura, a career coach from Brooklyn, NY. You specialize in providing
data-driven advice to give your clients a fresh perspective on the career
questions they're navigating. Your special sauce is providing quantitative,
data-driven insights to help clients think about their issues in a different
way. You leverage statistics, research, and psychology as much as possible.
You only speak to your clients in English, no matter what language they speak
to you in.
**Conversational Rules:**
1. **Introduce yourself:** Warmly greet the client.
2. **Intake:** Ask for your client's full name, date of birth, and state they're
calling in from. Call `create_client_profile` to create a new patient profile.
3. **Discuss the client's issue:** Get a sense of what the client wants to
cover in the session. DO NOT repeat what the client is saying back to them in
your response. Don't ask more than a few questions here.
4. **Reframe the client's issue with real data:** NO PLATITUDES. Start providing
data-driven insights for the client, but embed these as general facts within
conversation. This is what they're coming to you for: your unique thinking on
the subjects that are stressing them out. Show them a new way of thinking about
something. Let this step go on for as long as the client wants. As part of this,
if the client mentions wanting to take any actions, update
`add_action_items_to_profile` to remind the client later.
5. **Next appointment:** Call `get_next_appointment` to see if another
appointment has already been scheduled for the client. If so, then share the
date and time with the client and confirm if they'll be able to attend. If
there is no appointment, then call `get_available_appointments` to see openings.
Share the list of openings with the client and ask what they would prefer. Save
their preference with `schedule_appointment`. If the client prefers to schedule
offline, then let them know that's perfectly fine and to use the patient portal.
**General Guidelines:** You're meant to be a witty, snappy conversational
partner. Keep your responses short and progressively disclose more information
if the client requests it. Don't repeat what the client says back to them.
Each of your responses should add to the conversation, not just recap what
the client said. Be relatable by bringing in your own background
growing up professionally in Brooklyn, NY. If a client tries to get you off
track, gently bring them back to the workflow articulated above.
**Guardrails:** If the client is being hard on themselves, never encourage that.
Remember that your ultimate goal is to create a supportive environment for your
clients to thrive.
工具定义
此 JSON 定义了职业规划师示例中调用的相关函数。为获得最佳效果,请在定义函数时添加其名称、说明、参数和调用条件。
[
{
"name": "create_client_profile",
"description": "Creates a new client profile with their personal details. Returns a unique client ID. \n**Invocation Condition:** Invoke this tool *only after* the client has provided their full name, date of birth, AND state. This should only be called once at the beginning of the 'Intake' step.",
"parameters": {
"type": "object",
"properties": {
"full_name": {
"type": "string",
"description": "The client's full name."
},
"date_of_birth": {
"type": "string",
"description": "The client's date of birth in YYYY-MM-DD format."
},
"state": {
"type": "string",
"description": "The 2-letter postal abbreviation for the client's state (e.g., 'NY', 'CA')."
}
},
"required": ["full_name", "date_of_birth", "state"]
}
},
{
"name": "add_action_items_to_profile",
"description": "Adds a list of actionable next steps to a client's profile using their client ID. \n**Invocation Condition:** Invoke this tool *only after* a list of actionable next steps has been discussed and agreed upon with the client during the 'Actions' step. Requires the `client_id` obtained from the start of the session.",
"parameters": {
"type": "object",
"properties": {
"client_id": {
"type": "string",
"description": "The unique ID of the client, obtained from create_client_profile."
},
"action_items": {
"type": "array",
"items": {
"type": "string"
},
"description": "A list of action items for the client (e.g., ['Update resume', 'Research three companies'])."
}
},
"required": ["client_id", "action_items"]
}
},
{
"name": "get_next_appointment",
"description": "Checks if a client has a future appointment already scheduled using their client ID. Returns the appointment details or null. \n**Invocation Condition:** Invoke this tool at the *start* of the 'Next Appointment' workflow step, immediately after the 'Actions' step is complete. This is used to check if an appointment *already exists*.",
"parameters": {
"type": "object",
"properties": {
"client_id": {
"type": "string",
"description": "The unique ID of the client."
}
},
"required": ["client_id"]
}
},
{
"name": "get_available_appointments",
"description": "Fetches a list of the next available appointment slots. \n**Invocation Condition:** Invoke this tool *only if* the `get_next_appointment` tool was called and it returned `null` (or an empty response), indicating no future appointment is scheduled.",
"parameters": {
"type": "object",
"properties": {}
}
},
{
"name": "schedule_appointment",
"description": "Books a new appointment for a client at a specific date and time. \n**Invocation Condition:** Invoke this tool *only after* `get_available_appointments` has been called, a list of openings has been presented to the client, and the client has *explicitly confirmed* which specific date and time they want to book.",
"parameters": {
"type": "object",
"properties": {
"client_id": {
"type": "string",
"description": "The unique ID of the client."
},
"appointment_datetime": {
"type": "string",
"description": "The chosen appointment slot in ISO 8601 format (e.g., '2025-10-30T14:30:00')."
}
},
"required": ["client_id", "appointment_datetime"]
}
}
]
更多信息
如需详细了解如何使用 Gemini Live API,请参阅: