Ic-gateway issue ? Content-range header

Hey,
I’m facing right now with an issue with range header, (i used a random pictures i had, >2Mb here : https://4lfpx-eqaaa-aaaag-aujna-cai.raw.icp0.io/icrc3.png) for example, it was working before but for some reason it looks like the ic gateway is returning bad stuff on the range header.

curl -I "https://4lfpx-eqaaa-aaaag-aujna-cai.raw.icp0.io/icrc3.png" 2>&1
HTTP/2 200
content-length: 2000000
curl: (16) Invalid HTTP header field was received: frame type: 1, stream: 1, name: [content-length], value: [2535142]

We receive 2 content-length header, one wrong and one correct.
Can you help here?

Note that locally, it seems to work correctly according to what i heard.

1 Like

Hey @Gwojda,

I see the two headers you are mentioning:

(1) Set by the canister

content-length
2000000

(2) Set by the HTTP gateway

content-length
2535142

But to dig deeper, I need to understand better what you are doing. You are writing a canister that returns streaming response (partial assets with range headers), right? You set the content-length.

Thanks. It’s a storage canister that serve asset on http endpoint. you can check here : ic-storage-canister/canister/src at master · BitySA/ic-storage-canister · GitHub and here for how assets are served : ic-storage-canister/canister/src/queries/http_request.rs at master · BitySA/ic-storage-canister · GitHub. I’m 99% sure it was working before, i’m pretty sure i tested like few months ago with a big videos and a big pictures. No update on ic-asset libs btw since a long time.

any updates? Thanks Thanks

Hey @rbirkner,
Any update here ? I saw some new update has been done here, but it’s looks like the issue is still there, even after update on my side.

I just use ic-asset-certification, so if the issue is on canister side, i guess it’s related with an error here

Ok, i think i found the bug.
using xxx.icp0.io/xxx It’s working correctly. Using xxx.raw.icp0.io/xxx it’s not. The problem is here :
http-gateway/packages/ic-http-gateway/src/protocol/handler.rs at main · dfinity/http-gateway · GitHub

I guess we should do same as later in the code :

None => {
    for HeaderField(name, value) in &agent_response.headers {
        if !is_range_request
            && status_code == 206
            && (name.eq_ignore_ascii_case(http_header::CONTENT_RANGE.as_ref())
                || name.eq_ignore_ascii_case(http_header::CONTENT_LENGTH.as_ref()))
        {
            // skip copying
        } else {
            response_builder = response_builder.header(name.as_ref(), value.as_ref());
        }
    }
}

What do you think @rbirkner ?
I can open the pr if you want.

Edit : No certify assets streaming issue by Gwojda · Pull Request #61 · dfinity/http-gateway · GitHub

Hi @Gwojda

Sorry for the late reply! Thanks a lot for looking into it and coming up with a solution. At the moment, we just didn’t have the bandwidth to dig that deep.

We will review the PR and then make sure it gets deployed.