模型

TTSVoice module-attribute

TTSVoice = Literal[
    "alloy",
    "ash",
    "coral",
    "echo",
    "fable",
    "onyx",
    "nova",
    "sage",
    "shimmer",
]

用于 TTSModelSettings voice 枚举的可导出类型

TTSModelSettings dataclass

TTS 模型的设置。

源代码位于 src/agents/voice/model.py

@dataclass
class TTSModelSettings:
    """Settings for a TTS model."""

    voice: TTSVoice | None = None
    """
    The voice to use for the TTS model. If not provided, the default voice for the respective model
    will be used.
    """

    buffer_size: int = 120
    """The minimal size of the chunks of audio data that are being streamed out."""

    dtype: npt.DTypeLike = np.int16
    """The data type for the audio data to be returned in."""

    transform_data: (
        Callable[[npt.NDArray[np.int16 | np.float32]], npt.NDArray[np.int16 | np.float32]] | None
    ) = None
    """
    A function to transform the data from the TTS model. This is useful if you want the resulting
    audio stream to have the data in a specific shape already.
    """

    instructions: str = (
        "You will receive partial sentences. Do not complete the sentence just read out the text."
    )
    """
    The instructions to use for the TTS model. This is useful if you want to control the tone of the
    audio output.
    """

    text_splitter: Callable[[str], tuple[str, str]] = get_sentence_based_splitter()
    """
    A function to split the text into chunks. This is useful if you want to split the text into
    chunks before sending it to the TTS model rather than waiting for the whole text to be
    processed.
    """

    speed: float | None = None
    """The speed with which the TTS model will read the text. Between 0.25 and 4.0."""

voice class-attribute instance-attribute

voice: TTSVoice | None = None

TTS 模型要使用的语音。如果未提供，将使用相应模型的默认语音。

buffer_size class-attribute instance-attribute

buffer_size: int = 120

正在流式传输的音频数据的最小块大小。

dtype class-attribute instance-attribute

dtype: DTypeLike = int16

要返回的音频数据的的数据类型。

transform_data class-attribute instance-attribute

transform_data: (
    Callable[
        [NDArray[int16 | float32]], NDArray[int16 | float32]
    ]
    | None
) = None

用于转换 TTS 模型数据的函数。如果您希望结果音频流已经以特定形状的数据呈现，这将很有用。

instructions 类属性 实例属性

instructions: str = "You will receive partial sentences. Do not complete the sentence just read out the text."

用于 TTS 模型的指令。如果您希望控制音频输出的语调，这将很有用。

text_splitter class-attribute instance-attribute

text_splitter: Callable[[str], tuple[str, str]] = (
    get_sentence_based_splitter()
)

用于将文本拆分为块的函数。如果您希望在将其发送到 TTS 模型之前将文本拆分为块，而不是等待处理整个文本，这将很有用。

speed class-attribute instance-attribute

speed: float | None = None

TTS 模型阅读文本的速度。介于 0.25 和 4.0 之间。

TTSModel

基础: ABC

可以将文本转换为音频输出的文本到语音模型。

源代码位于 src/agents/voice/model.py

class TTSModel(abc.ABC):
    """A text-to-speech model that can convert text into audio output."""

    @property
    @abc.abstractmethod
    def model_name(self) -> str:
        """The name of the TTS model."""
        pass

    @abc.abstractmethod
    def run(self, text: str, settings: TTSModelSettings) -> AsyncIterator[bytes]:
        """Given a text string, produces a stream of audio bytes, in PCM format.

        Args:
            text: The text to convert to audio.

        Returns:
            An async iterator of audio bytes, in PCM format.
        """
        pass

model_name abstractmethod property

model_name: str

TTS 模型的名称。

run abstractmethod

run(
    text: str, settings: TTSModelSettings
) -> AsyncIterator[bytes]

给定一个文本字符串，生成 PCM 格式的音频字节流。

参数

名称	类型	描述	默认
文本	str	要转换为音频的文本。	required

返回值

类型	描述
AsyncIterator[bytes]	PCM 格式的音频字节的异步迭代器。

源代码位于 src/agents/voice/model.py

@abc.abstractmethod
def run(self, text: str, settings: TTSModelSettings) -> AsyncIterator[bytes]:
    """Given a text string, produces a stream of audio bytes, in PCM format.

    Args:
        text: The text to convert to audio.

    Returns:
        An async iterator of audio bytes, in PCM format.
    """
    pass

StreamedTranscriptionSession

基础: ABC

音频输入的流式转录。

源代码位于 src/agents/voice/model.py

class StreamedTranscriptionSession(abc.ABC):
    """A streamed transcription of audio input."""

    @abc.abstractmethod
    def transcribe_turns(self) -> AsyncIterator[str]:
        """Yields a stream of text transcriptions. Each transcription is a turn in the conversation.

        This method is expected to return only after `close()` is called.
        """
        pass

    @abc.abstractmethod
    async def close(self) -> None:
        """Closes the session."""
        pass

transcribe_turns abstractmethod

transcribe_turns() -> AsyncIterator[str]

生成文本转录的流。每个转录都是对话中的一轮。

只有在调用 close() 后，此方法才应返回。

源代码位于 src/agents/voice/model.py

@abc.abstractmethod
def transcribe_turns(self) -> AsyncIterator[str]:
    """Yields a stream of text transcriptions. Each transcription is a turn in the conversation.

    This method is expected to return only after `close()` is called.
    """
    pass

close abstractmethod async

close() -> None

关闭会话。

源代码位于 src/agents/voice/model.py

@abc.abstractmethod
async def close(self) -> None:
    """Closes the session."""
    pass

STTModelSettings dataclass

语音到文本模型的设置。

源代码位于 src/agents/voice/model.py

@dataclass
class STTModelSettings:
    """Settings for a speech-to-text model."""

    prompt: str | None = None
    """Instructions for the model to follow."""

    language: str | None = None
    """The language of the audio input."""

    temperature: float | None = None
    """The temperature of the model."""

    turn_detection: dict[str, Any] | None = None
    """The turn detection settings for the model when using streamed audio input."""

prompt 类属性 实例属性

prompt: str | None = None

模型要遵循的指令。

language class-attribute instance-attribute

language: str | None = None

音频输入的语言。

temperature class-attribute instance-attribute

temperature: float | None = None

模型的温度。

turn_detection class-attribute instance-attribute

turn_detection: dict[str, Any] | None = None

在使用流式音频输入时，模型的回合检测设置。

STTModel

基础: ABC

可以将音频输入转换为文本的语音到文本模型。

源代码位于 src/agents/voice/model.py

class STTModel(abc.ABC):
    """A speech-to-text model that can convert audio input into text."""

    @property
    @abc.abstractmethod
    def model_name(self) -> str:
        """The name of the STT model."""
        pass

    @abc.abstractmethod
    async def transcribe(
        self,
        input: AudioInput,
        settings: STTModelSettings,
        trace_include_sensitive_data: bool,
        trace_include_sensitive_audio_data: bool,
    ) -> str:
        """Given an audio input, produces a text transcription.

        Args:
            input: The audio input to transcribe.
            settings: The settings to use for the transcription.
            trace_include_sensitive_data: Whether to include sensitive data in traces.
            trace_include_sensitive_audio_data: Whether to include sensitive audio data in traces.

        Returns:
            The text transcription of the audio input.
        """
        pass

    @abc.abstractmethod
    async def create_session(
        self,
        input: StreamedAudioInput,
        settings: STTModelSettings,
        trace_include_sensitive_data: bool,
        trace_include_sensitive_audio_data: bool,
    ) -> StreamedTranscriptionSession:
        """Creates a new transcription session, which you can push audio to, and receive a stream
        of text transcriptions.

        Args:
            input: The audio input to transcribe.
            settings: The settings to use for the transcription.
            trace_include_sensitive_data: Whether to include sensitive data in traces.
            trace_include_sensitive_audio_data: Whether to include sensitive audio data in traces.

        Returns:
            A new transcription session.
        """
        pass

model_name abstractmethod property

model_name: str

STT 模型的名称。

transcribe abstractmethod async

transcribe(
    input: AudioInput,
    settings: STTModelSettings,
    trace_include_sensitive_data: bool,
    trace_include_sensitive_audio_data: bool,
) -> str

给定一个音频输入，生成文本转录。

参数

名称	类型	描述	默认
input	AudioInput	要转录的音频输入。	required
settings	STTModelSettings	用于转录的设置。	required
trace_include_sensitive_data	bool	是否在跟踪中包含敏感数据。	required
trace_include_sensitive_audio_data	bool	是否在跟踪中包含敏感音频数据。	required

返回值

类型	描述
str	音频输入的文本转录。

源代码位于 src/agents/voice/model.py

@abc.abstractmethod
async def transcribe(
    self,
    input: AudioInput,
    settings: STTModelSettings,
    trace_include_sensitive_data: bool,
    trace_include_sensitive_audio_data: bool,
) -> str:
    """Given an audio input, produces a text transcription.

    Args:
        input: The audio input to transcribe.
        settings: The settings to use for the transcription.
        trace_include_sensitive_data: Whether to include sensitive data in traces.
        trace_include_sensitive_audio_data: Whether to include sensitive audio data in traces.

    Returns:
        The text transcription of the audio input.
    """
    pass

create_session abstractmethod async

create_session(
    input: StreamedAudioInput,
    settings: STTModelSettings,
    trace_include_sensitive_data: bool,
    trace_include_sensitive_audio_data: bool,
) -> StreamedTranscriptionSession

创建一个新的转录会话，您可以将音频推送到该会话，并接收文本转录的流。

参数

名称	类型	描述	默认
input	StreamedAudioInput	要转录的音频输入。	required
settings	STTModelSettings	用于转录的设置。	required
trace_include_sensitive_data	bool	是否在跟踪中包含敏感数据。	required
trace_include_sensitive_audio_data	bool	是否在跟踪中包含敏感音频数据。	required

返回值

类型	描述
StreamedTranscriptionSession	一个新的转录会话。

源代码位于 src/agents/voice/model.py

@abc.abstractmethod
async def create_session(
    self,
    input: StreamedAudioInput,
    settings: STTModelSettings,
    trace_include_sensitive_data: bool,
    trace_include_sensitive_audio_data: bool,
) -> StreamedTranscriptionSession:
    """Creates a new transcription session, which you can push audio to, and receive a stream
    of text transcriptions.

    Args:
        input: The audio input to transcribe.
        settings: The settings to use for the transcription.
        trace_include_sensitive_data: Whether to include sensitive data in traces.
        trace_include_sensitive_audio_data: Whether to include sensitive audio data in traces.

    Returns:
        A new transcription session.
    """
    pass

VoiceModelProvider

基础: ABC

语音模型提供商的基接口。

模型提供商负责根据名称创建语音到文本和文本到语音模型。

源代码位于 src/agents/voice/model.py

class VoiceModelProvider(abc.ABC):
    """The base interface for a voice model provider.

    A model provider is responsible for creating speech-to-text and text-to-speech models, given a
    name.
    """

    @abc.abstractmethod
    def get_stt_model(self, model_name: str | None) -> STTModel:
        """Get a speech-to-text model by name.

        Args:
            model_name: The name of the model to get.

        Returns:
            The speech-to-text model.
        """
        pass

    @abc.abstractmethod
    def get_tts_model(self, model_name: str | None) -> TTSModel:
        """Get a text-to-speech model by name."""

get_stt_model abstractmethod

get_stt_model(model_name: str | None) -> STTModel

按名称获取语音转文本模型。

参数

名称	类型	描述	默认
模型名称	str | None	The name of the model to get.	required

返回值

类型	描述
STTModel	语音转文本模型。

源代码位于 src/agents/voice/model.py

@abc.abstractmethod
def get_stt_model(self, model_name: str | None) -> STTModel:
    """Get a speech-to-text model by name.

    Args:
        model_name: The name of the model to get.

    Returns:
        The speech-to-text model.
    """
    pass

get_tts_model abstractmethod

get_tts_model(model_name: str | None) -> TTSModel

按名称获取文本转语音模型。

源代码位于 src/agents/voice/model.py

@abc.abstractmethod
def get_tts_model(self, model_name: str | None) -> TTSModel:
    """Get a text-to-speech model by name."""