# Boundary node http response headers

**URL:** <https://forum.dfinity.org/t/boundary-node-http-response-headers/10747>\
**Category:** Developers\
**Created:** [February 7, 2022, 4:21pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747 "2022-02-07T16:21:52Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![jplevyak](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/jplevyak/32/3299_2.png) [@jplevyak](https://forum.dfinity.org/u/jplevyak)\
**Post date:** [February 10, 2022, 10:00pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/42 "2022-02-10T22:00:57Z")

</div>

Thanx for the pointer! I had wondered if rolling hashes or something could do it. We want to have certification checking by default with opt out for raw.ic0.app via icx-proxy where the opt out would itself be certified. We could pass the headers but currently there is no way to certify the Range response. That said I don’t think there is a security issue, but I need to get it vetted.

---

<div class="post-metadata">

**Author:** ![paulyoung](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/paulyoung/32/5251_2.png) [@paulyoung](https://forum.dfinity.org/u/paulyoung)\
**Post date:** [February 11, 2022, 12:25am UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/43 "2022-02-11T00:25:39Z")

</div>

> [@jplevyak](#):
>
> We have an automatic chunking system for uploads and downloads and we can use that for reading certified chunks in the service worker and icx-proxy

This sounds like exactly what I need.

---

<div class="post-metadata">

**Author:** ![3cL1p5e7](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/3cl1p5e7/32/4649_2.png) [@3cL1p5e7](https://forum.dfinity.org/u/3cL1p5e7)\
**Post date:** [February 11, 2022, 8:57pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/44 "2022-02-11T20:57:53Z")

</div>

HI, @nomeata 🙂  
Why is the SHA-256 hash stored for each asset and validated on service-worker?  
I’m not expert, but isn’t it enough that the service worker checks the IC-certificates in the headers of asset (to avoid data spoofing) for “certification”?  
For example about sha: if I load assets myself with manually generated md5-hash, then hash saved on certified-assets canister, but service-worker validation will fail.

---

<div class="post-metadata">

**Author:** ![nomeata](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/nomeata/32/733_2.png) [@nomeata](https://forum.dfinity.org/u/nomeata)\
**Post date:** [February 11, 2022, 9:32pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/45 "2022-02-11T21:32:34Z")

</div>

I don’t quite understand the question. The certification needs to build a trust path between something that the user (the service work) knows, namely the IC root key, and the file just loaded. This involves a few steps (root public key → subnet public key → subnet state merkle tree root → subnet state merkle tree entry with certified data from canister → canister merkle tree root → canister merkle tree entry with sha256 of file loaded → file), and one of the steps requires the SHA256 signature of the file. So the canister needs the SHA256. Also see this video for more details:

[![](https://us1.discourse-cdn.com/flex023/uploads/dfn/original/2X/b/bd8d2cffd9c4f56b6c47f153924fec22d570ab3a.jpeg "Inside the Internet Computer | Certified Variables") ](https://www.youtube.com/watch?v=mZbFhRIHIiY)

Is it the case that the default asset canister doesn’t calculate the SHA256 upon upload, but requires the uploader to set it? Then that’s an engineering choice around that canister.

---

<div class="post-metadata">

**Author:** ![3cL1p5e7](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/3cl1p5e7/32/4649_2.png) [@3cL1p5e7](https://forum.dfinity.org/u/3cL1p5e7)\
**Post date:** [February 11, 2022, 9:48pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/46 "2022-02-11T21:48:03Z")

</div>

Now everything is clear for me, thanks a lot!

---

<div class="post-metadata">

**Author:** ![3cL1p5e7](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/3cl1p5e7/32/4649_2.png) [@3cL1p5e7](https://forum.dfinity.org/u/3cL1p5e7)\
**Post date:** [February 11, 2022, 10:03pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/47 "2022-02-11T22:03:47Z")

</div>

> [@nomeata](#):
>
> There are certification schemes that support that, using suitable hash functions, and we did consider them back then before we settled on this certification MVP because we needed something simple. But I wouldn’t say it’s impossible, we could extend our protocol here. See, e.g.

Do you have any plans for this in the near future?

---

<div class="post-metadata">

**Author:** ![nomeata](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/nomeata/32/733_2.png) [@nomeata](https://forum.dfinity.org/u/nomeata)\
**Post date:** [February 11, 2022, 10:33pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/48 "2022-02-11T22:33:05Z")

</div>

> [@3cL1p5e7](#):
>
> Do you have any plans for this in the near future?

I don’t.

BTW, for many applications it might be a good option to serve HTML and JavaScript safely over the certified URL, and then do more complicated stuff (e.g. fetch video chunks certified by a stream signature) from your JS in an application-specific way, and _not_ using the certified HTTP stuff in that case. This way no need to wait for me or DFINITY to find generic solutions (which tend to be harder).

---

<div class="post-metadata">

**Author:** ![faraz.shaikh](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/faraz.shaikh/32/4005_2.png) [@faraz.shaikh](https://forum.dfinity.org/u/faraz.shaikh)\
**Post date:** [February 12, 2022, 12:24am UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/49 "2022-02-12T00:24:30Z")

</div>

Guys this is a humble opinion on two things that I feel are faces of the same decentralization coin.

1. \_raw  
This is a security hole I wish Dfinity as org had never introduced. The implication of a single replica serving arbitrary data that cannot be checked for integrity is difficult to digest. Also now that \_raw is deployed in the wild it’s difficult to roll back such decisions.

2. http header  
My opinion is that Dfinity should NOT provide guidance/spec out “web content related” canisters. HTTP is an independent spec, what does mean to have a restrictive/additive HTTP spec within IC spec. It’s a slippery slope why stop at webserver specs, lets’s also do mail server specs

I sympathize with @lastmjs, he is being coerced into doing streaming “a” particular way. Having said that I don’t think adding anything to spec or boundary nodes is the right idea.

Instead, let’s work towards some solution where  
@lastjms can run his canister, and **his** boundary nodes and expose **any** HEADER, any un/secure content he wants to. There is dissonance in what we are asking developers to do. Providing an unsecured \_raw path, having an opinion on streaming be done X vs Y way is not an IC concern.

The effort required may be similar.

---

<div class="post-metadata">

**Author:** ![3cL1p5e7](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/3cl1p5e7/32/4649_2.png) [@3cL1p5e7](https://forum.dfinity.org/u/3cL1p5e7)\
**Post date:** [February 12, 2022, 8:54pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/50 "2022-02-12T20:54:35Z")

</div>

I agree with your words!

About your suggestion: there are some limitations for boundary node:

- The deploing own boundary node under custom domain is might be a great solution. But then it will lose the ability to work under `ic0.app` domain, especially when custom subdomains are released. Demergence-podcasts will live fine on `demergence.com` (for example), but not on potentially `demergence.ic0.app`.
- This may can affect the audience’s trust (not a fact)
- The streaming feature needs not only for demergence, it might be useful for others. And the **easiest** way to get streaming feature for ordinary developer is to use `certified_assets` canister (simple set `"type"="assets"` in your `dfx.json` and lets gooo)

In general, a solution is acceptable. Moreover, there is an [info about deploy boundary node](https://forum.dfinity.org/t/btcflower-xyz-seems-to-be-hosted-on-ic-but-why-this-domain-name-does-not-need-redirection/10787/4)

@lastmjs  
I’ve been thinking about this for the last week and can suggest the next:

- In theory, you can to develop custom service-worker with range streaming support and register it on your canister manually. It will catch `.raw.` requests with `range` headers, will try to collect chunks from canister query methods and will return response with ranges to the client. [There is my attempts](https://github.com/dfinity/ic/compare/master...3cL1p5e7:feature/service-worker-streaming-chunks) to do this and it works **nearly** fine with TODOs for `certfied_assets`

---

<div class="post-metadata">

**Author:** ![skilesare](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/skilesare/32/5609_2.png) [@skilesare](https://forum.dfinity.org/u/skilesare)\
**Post date:** [April 23, 2022, 5:05pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/51 "2022-04-23T17:05:16Z")

</div>

To whom it may concern: I’ve been working with Daniel to try to figure out this strange safari issue where I can’t get our video to play from our canister. If I run a local version of the icx-proxy and point it at ic0.app I can get my video to play at [http://localhost:3000/-/1/ex?canisterId=r5m5i-tiaaa-aaaaj-acgaq-cai](http://localhost:3000/-/1/ex?canisterId=r5m5i-tiaaa-aaaaj-acgaq-cai) but if I call the same thing at [https://r5m5i-tiaaa-aaaaj-acgaq-cai.raw.ic0.app/-/1/ex](https://r5m5i-tiaaa-aaaaj-acgaq-cai.raw.ic0.app/-/1/ex) the video does not play.

In both cases, safari stops the request when it sees ‘video’ in the content type and sends a new range request asking for the first two bytes. When I go through a local proxy that range request gets through(It is a header on the request (Range: bytes=0-1). When I go straight to raw.ic0.app Safari says that it is sending the header, but i’ve manipulated my server to return the actual headers that I’m getting and the range request is not there. So either Safari is lying(Daniel thinks so) or something is stripping off the range request on the way to my canister. I don’t know the topgraphy of where a request actually goes, but I suspect there is a boundary node and a proxy?

```auto
Request - through local version of icx-proxy
GET /-/1/ex HTTP/1.1
Accept: */*
Connection: Keep-Alive
Range: bytes=0-1
Host: localhost:3000
Accept-Language: en-US,en;q=0.9
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15
Referer: http://localhost:3000/-/1/ex?canisterId=r5m5i-tiaaa-aaaaj-acgaq-cai
Accept-Encoding: identity
X-Playback-Session-Id: 6FFA5EE3-52F6-4DDC-8297-300A4C22A9DF

Response
HTTP/1.1 206 Partial Content
Content-Range: bytes 0-1/46984888
Accept-Ranges: bytes
Content-Type: video/mp4
Content-Length: 2
Date: Thu, 21 Apr 2022 22:58:16 GMT

Request 2 - through raw.ic0.app
GET /-/1/ex
Range: bytes=0-1
Accept: */*
Referer: https://r5m5i-tiaaa-aaaaj-acgaq-cai.raw.ic0.app/-/1/ex
Accept-Encoding: identity
Connection: Keep-Alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15
X-Playback-Session-Id: 62FA3A12-FD2A-4C34-839A-B6963DCFC6F1

Actual headers that get to my canister
[("host", "r5m5i-tiaaa-aaaaj-acgaq-cai.raw.ic0.app"), 
("x-real-ip", "108.230.245.93"), 
("x-forwarded-for", "108.230.245.93"), 
("x-forwarded-proto", "https"), ("connection", "close"), 
("user-agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.0 Safari/605.1.15"), ("accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"),
 ("accept-language", "en-US,en;q=0.9"), 
("accept-encoding", "gzip, deflate, be")]

Response:
200
Access-Control-Allow-Credentials: true
Content-Type: video/mp4
Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS
Access-Control-Expose-Headers: Content-Length,Content-Range
Access-Control-Allow-Origin: *
Date: Thu, 21 Apr 2022 22:56:06 GMT
Access-Control-Allow-Headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range,Cookie
Server: nginx/1.21.3
x-cache-status: MISS

```

This thread confirms my suspicions.

Looks like the short term solution is to set up a server running icx-proxy that can server your requests. So @lastmjs you could set up [stream.demergence.com/episode-tag?canisgerId=xxxxxxxxxxxxx](http://stream.demergence.com/episode-tag?canisgerId=xxxxxxxxxxxxx) and it would work.

As far as certified assets go, range requests allow you to return the results you want vs. what was requested, so if you certified 1MB chunks you could always return at least that chunk. If it requests something in the middle of a chunk, rewind to the earliest certified starting point and return the chunk. maybe the chunks need to be smaller, but since safari requests them in order and the latency is pretty bad it can take along time to download large files.

A bit irrelevant as it seems like icx-proxy does not certify data anyway (but it would be great to have confirmation at [Can the icx-proxy return certified assets?](https://forum.dfinity.org/t/can-the-icx-proxy-return-certified-assets/12358)

---

<div class="post-metadata">

**Author:** ![lastmjs](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/lastmjs/32/3406_2.png) [@lastmjs](https://forum.dfinity.org/u/lastmjs)\
**Post date:** [April 23, 2022, 5:26pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/52 "2022-04-23T17:26:35Z")

</div>

The boundary nodes are most likely removing the Range header, that’s exactly the issue I was having

---

<div class="post-metadata">

**Author:** ![faraz.shaikh](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/faraz.shaikh/32/4005_2.png) [@faraz.shaikh](https://forum.dfinity.org/u/faraz.shaikh)\
**Post date:** [April 23, 2022, 5:27pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/53 "2022-04-23T17:27:57Z")

</div>

> <https://github.com/dfinity/ic/blob/master/ic-os/boundary-guestos/rootfs/etc/nginx/conf.d/001-mainnet-nginx.conf>

Line 390

This is _not_ the nginx configuration on the boundary nodes but its a exact copy. You can check how the boundary is configured with respect to header stripping by looking at this code.

---

<div class="post-metadata">

**Author:** ![skilesare](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/skilesare/32/5609_2.png) [@skilesare](https://forum.dfinity.org/u/skilesare)\
**Post date:** [April 23, 2022, 5:47pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/54 "2022-04-23T17:47:29Z")

</div>

So if it isn’t being stripped, do you think that safari is just not sending it properly?

---

<div class="post-metadata">

**Author:** ![faraz.shaikh](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/faraz.shaikh/32/4005_2.png) [@faraz.shaikh](https://forum.dfinity.org/u/faraz.shaikh)\
**Post date:** [April 23, 2022, 5:48pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/55 "2022-04-23T17:48:57Z")

</div>

If it’s getting stripped it doesn’t matter how safari is sending it. It will get stripped

---

<div class="post-metadata">

**Author:** ![skilesare](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/skilesare/32/5609_2.png) [@skilesare](https://forum.dfinity.org/u/skilesare)\
**Post date:** [April 23, 2022, 6:21pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/56 "2022-04-23T18:21:06Z")

</div>

Implementing our own version of a ICX proxy should fix this issue, correct?

---

<div class="post-metadata">

**Author:** ![faraz.shaikh](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/faraz.shaikh/32/4005_2.png) [@faraz.shaikh](https://forum.dfinity.org/u/faraz.shaikh)\
**Post date:** [April 23, 2022, 6:49pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/57 "2022-04-23T18:49:36Z")

</div>

@skilesare no, the next step for the issue you are facing would be

-what header is being stripped and where (in the nginx source code i shared)  
-motivate inclusion of the header/or workaround the header.

Currently AFK, would love to get your problem resolved under the video streaming effort already underway at Dfinity. there are many ways to do video streaming - the core focus is to select one that doesn’t break boundary node caching

Boundary node caching is absolute must if the IC is to host YouTube scale videos. So it’s not just about simply enabling range headers. What’s your project/contact point I will try to get you plugged into the video effort

---

<div class="post-metadata">

**Author:** ![skilesare](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/skilesare/32/5609_2.png) [@skilesare](https://forum.dfinity.org/u/skilesare)\
**Post date:** [April 23, 2022, 6:59pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/58 "2022-04-23T18:59:26Z")

</div>

Well…I have it mostly solved for my purposes through a local icx-proxy. We’re going to deploy it to a domain early next week and see if it continues to work.

You can simulate your self by running a local icx proxy and pointing it to ic0.app and hitting that same url from safari. It does some strange things for that video about 30MB in, but full video ends up coming down.

Added you to an on going conversion.

---

<div class="post-metadata">

**Author:** ![lastmjs](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/lastmjs/32/3406_2.png) [@lastmjs](https://forum.dfinity.org/u/lastmjs)\
**Post date:** [May 14, 2022, 2:58pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/59 "2022-05-14T14:58:31Z")

</div>

Can we get an update from DFINITY on when boundary nodes will allow for range requests? I would also love to know the current strategy for supporting streaming audio/video and when we can expect this all to be released.

My podcast Demergence is hosted 100% on the IC but its streaming has been broken since the beginning since there is no support for range requests. I would to get this resolved soon so we can have full podcast support on the IC!

---

<div class="post-metadata">

**Author:** ![skilesare](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/skilesare/32/5609_2.png) [@skilesare](https://forum.dfinity.org/u/skilesare)\
**Post date:** [May 14, 2022, 11:27pm UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/60 "2022-05-14T23:27:10Z")

</div>

Range requests work through icx proxy.

See: [https://nft.origyn.network/-/nftforgood\_uffc/-/ogy.nftforgood\_uffc.1/social](https://nft.origyn.network/-/nftforgood_uffc/-/ogy.nftforgood_uffc.1/social)

Even works in safari for the most part.

---

<div class="post-metadata">

**Author:** ![lastmjs](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/lastmjs/32/3406_2.png) [@lastmjs](https://forum.dfinity.org/u/lastmjs)\
**Post date:** [May 15, 2022, 2:01am UTC](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747/61 "2022-05-15T02:01:42Z")

</div>

What about on a regular IC subdomain with no proxy?

[Previous page](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747.md?page=2)

[Next page](https://forum.dfinity.org/t/boundary-node-http-response-headers/10747.md?page=4)
