Skip to content

HTTP.download() does not automatically decode Content-Encoding #889

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
staticfloat opened this issue Jul 19, 2022 · 1 comment · Fixed by #986
Closed

HTTP.download() does not automatically decode Content-Encoding #889

staticfloat opened this issue Jul 19, 2022 · 1 comment · Fixed by #986

Comments

@staticfloat
Copy link
Contributor

Verisons

Julia v1.7.3
HTTP.jl v1.2.0
MbedTLS.jl v1.1.1

Issue

When calling HTTP.get() on a resource with Content-Encoding: gzip, the response is automatically decoded for me:

julia> using HTTP
       io = IOBuffer()
       r = HTTP.get("https://httpbin.org/gzip", response_stream=io);
       println(String(take!(io)))
{
  "gzipped": true, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip", 
    "Content-Length": "0", 
    "Host": "httpbin.org", 
    "User-Agent": "HTTP.jl/1.7.3", 
    "X-Amzn-Trace-Id": "Root=1-62d6f292-06e09add2b03b541490576d4"
  }, 
  "method": "GET", 
  "origin": "75.172.111.107"
}

Using HTTP.download(), it does not:

julia> using HTTP
       HTTP.download("https://httpbin.org/gzip", "/dev/stdout"; update_period=Inf);
���b�=��j�0�w?��*Œ]�:��B�R\�*[gY���|]���x����ݵ(Kb.vY@���<��͠4�5�k#���3��wd
                                                                         eJ�n��:�tj���;����9�*��_��q�c>���3q5%N}��~~��IV��7��G��F�o�|��=>s�
�QZL��u�&C����p$6�g�-��{7������K�8�W����Sj2
@Heptazhou
Copy link

Heptazhou commented Jul 24, 2022

As far as I found, this is a regression introduced since v1.0.0, for files that could be downloaded correctly without compression

the thing is:

  1. Auto decompress gzip-encoding response bodies #838 implemented gzip decompression for HTTP.get but not for HTTP.download
  2. Start filling out client.md, websockets.md, and server.md doc guides #851 set "Accept-Encoding" => "gzip" as default header (R44-R46)

it works for me immediately by either:

  1. simply run pkg> pin HTTP@0 to use v0.9.17
  2. counterintuitively though, use HTTP.download(decompress = false, args...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants