Expand description
This module provides functionality for working with streaming completion models. It provides traits and types for generating streaming completion requests and handling streaming completion responses.
The main traits defined in this module are:
- StreamingPrompt: Defines a high-level streaming LLM one-shot prompt interface
- StreamingChat: Defines a high-level streaming LLM chat interface with history
- StreamingCompletion: Defines a low-level streaming LLM completion interface
- StreamingCompletionModel: Defines a streaming completion model interface
Enums§
- Streaming
Choice - Enum representing a streaming chunk from the model
Traits§
- Streaming
Chat - Trait for high-level streaming chat interface
- Streaming
Completion - Trait for low-level streaming completion interface
- Streaming
Completion Model - Trait defining a streaming completion model
- Streaming
Prompt - Trait for high-level streaming prompt interface
Functions§
- stream_
to_ stdout - helper function to stream a completion request to stdout