Download a large file w/o loading it into memory


#1

So I am able to have an endpoint to download a file:

import os
from apistar import http

def download_file(id_: int) -> http.Response:
     """Download a file identified by some id"""
     path = get_path_from_id(id_)
     filename = get_fliename_from_id(id_)
     content_type = get_content_type_from_id(id_)
     with open(path, "rb") as fd:
        return http.Response(
            content=fd.read(),  # <-- file read into memory
            status_code=200,
            headers={ 
                "Content-Type": content_type,
                "Content-Length": os.path.getsize(path)
                "Content-Disposition": f'attachment; filename="{filename}"',
                "Connection": "keep-alive",
            },
        )

And this while works, read the entire file into memory with content=fd.read(). This can be a problem when I need to serve large files. Note that the content I wish to serve is not static.

I tried to override http.Response having:

class DownloadResponse(http.Response):

    def render(self, content: io) -> io:
        return iter(lambda: content.read(CHUNK_SIZE), b'')

Or any other permutation of it and using it with:

def download_file(id_: int) -> DownloadResponse:
     # ...
     with open(path, "rb") as fd:
        return DownloadResponse(
            content=fd,  
            #  ...

But this boils down to
b''.join(wsgi_response)) where wsgi_response was expected to be a list of byte-strings and not a list of iterables.

I also looked at apistar.codecs.download.DownloadCodec but as far as I could understand the code it allows ApiStar to download a file itself, not let the user download a file. I accept the possibility that I don’t understand this at all, in this case I don’t know how should I use DownloadCodec.

Is what I am trying to achieve supported by ApiStar (as an old fashioned WSGI app) at all? If so any pointer will be greatly appreciated.

See also this related post.