stream and astream
stream() returns a normal iterator. astream() returns an async iterator. A chat model usually yields AIMessageChunk objects that can be added together into one message.
Tiny example
Section titled “Tiny example”full = None
async for chunk in model.astream("Draft a short incident update"): full = chunk if full is None else full + chunk await websocket.send_text(chunk.text)
print(full.text)Streaming is a pipeline property
Section titled “Streaming is a pipeline property”- The provider must send chunks.
- Each middle step must preserve them.
- The transport must flush them to the client.
- Your application must still record the final result and failures.
The base Runnable implementation of stream() calls invoke() and the base astream() calls ainvoke(). A component must override that behavior to produce real incremental output.
Failure note
Section titled “Failure note”A partial answer may already be visible when a later tool or network call fails. Design a terminal error event, cancellation handling, and UI state for incomplete output.