Support Response Streaming

[limcheekin]

Hi there,

Did the runtime interface client support streaming response as per announced?
https://aws.amazon.com/blogs/compute/introducing-aws-lambda-response-streaming/

If no, appreciate your sharing is there any near-term plan to work on it?

Thank you.

[devesch]

I'd like to see the streaming response in python too!!

[pprados]

With tools like LLM, the support is important. Else, it's not possible to use a lambda to return a stream of text.

[nicolasvo]

@pprados aws-lambda-web-adapter enables response streaming from a python lambda.

I implemented an example: https://github.com/nicolasvo/ollama-lambda/tree/master/lambda (see Dockerfile.fastapi and
main.py).

However, this is done without lambda function handler. So if you need it, it might not be a solution.

[limcheekin]

@nicolasvo Thanks for sharing. Unfortunately I don't have time to look into it now as I'm working on higher priorities task.