yield data, which is sent downstream via the text/event-stream Content-Type.
You can send data in JSON format and decode it on the client side.
A minimal example:
Endpoints
Streaming Endpoints
Stream live model output from a Cerebrium endpoint over server-sent events by yielding results from a Python generator or iterator function.
Streaming sends live output from a model over a server-sent event (SSE) stream.
It works with any Python object that implements the iterator or generator protocol.
The generator/iterator must
Deploy this snippet and call the endpoint. SSE events appear progressively, one per second:
This should output:
The remaining data streams in every second:
Postman also supports SSE streams natively.
For a Falcon-7B streaming example, see the streaming endpoint example.