Hi guys,
I just updated my http_request tests, and since i updated this, once i’m serving asset (using http-certify-asset) which is a big one (send by batch), i have this error :
called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Request, url: "http://uqqxf-5h777-77774-qaaaa-cai.raw.localhost:55697/test.png", source: hyper_util::client::legacy::Error(SendRequest, hyper::Error(IncompleteMessage)) }
Basicly what i’m doing is this :
let endpoint = pic.make_live(None);
println!("gateway url: {:?}", endpoint);
let mut url = endpoint.clone();
let client: reqwest::blocking::Client = ClientBuilder::new()
.resolve(
"uqqxf-5h777-77774-qaaaa-cai.raw.localhost",
SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), url.port().unwrap())
)
.redirect(reqwest::redirect::Policy::none())
.build()
.unwrap();
let gateway_host = endpoint.host().unwrap();
let host = format!("{}.raw.{}", storage_canister_id, gateway_host);
url.set_host(Some(&host)).unwrap();
url.set_path("/test.png");
let mut received_bytes = Vec::new();
let mut start = 0;
let mut chunk_size = 1024 * 1024; // Default chunk size
let max_retries = 5;
loop {
// Initial request to get the chunk size from the headers
println!("url: {:?}", url);
let initial_res = client.get(url.clone()).send().unwrap();
if initial_res.status() == 206 {
if let Some(content_range) = initial_res.headers().get("content-range") {
let content_range_str = content_range.to_str().unwrap();
if let Some(range) = content_range_str.split('/').next() {
println!("content-range: {:?}", range);
let parts: Vec<&str> = range.split(' ').nth(1).unwrap().split('-').collect();
println!("parts: {:?}", parts);
if parts.len() == 2 {
let start_range: usize = parts[0].parse().unwrap();
let end_range: usize = parts[1].parse().unwrap();
chunk_size = end_range - start_range + 1;
}
}
}
break;
} else {
panic!("Failed to get initial response: {:?}", initial_res);
}
}
And the let initial_res = client.get(url.clone()).send().unwrap(); fail.
Sorry if I misunderstood the question. It seems reasonable that for a large batched asset we need to specify the range. Otherwise, the canister tries to send the entire asset and fails, right?
The right way to do it should :
if first call return a 206 http response, then you start to do the calls using Range Headers. That’s how my tests was working previously with pocket-ic.
Actually, the previously first http query which was returning a 206, now return an error, unless i add the Range Headers directly, and it’s only working if i put 2000000 as batch size.
The PocketIC server v7.0.0 used the HTTP gateway v0.1.0-b0 while the latest PocketIC server v8.0.0 uses the HTTP gateway v0.2.0. Maybe @NathanosDev could please comment on whether there are expected behavioral changes?
We are using ic-asset-certification, but if we do not provide an Range parameters in the header at the first call, calls fail to get a big files.
Previously, we were doing a first http calls, and if the request was returning a 206 status code, we were adding the necessary call with the Range headers setted correctly.
Why this comportment changed ? What is the right way now to work with the HTTP gateway v0.2.0 ?
You don’t need to add range parameters. You should just make a standard GET request to the HTTP Gateway and you will get back a 200 response with the full asset, don’t worry about the streaming. The HTTP Gateway handles that for you.
As a sidenote, since you’re using ic-asset-certification then you also don’t need to use raw. The responses are certified.
i’m updating our tests to follow that method of testing, but i’m currently facing an error with upgrade calls.
I have a case where i need to do an upgrade call, and so i’m returning in my http_request return HttpResponse::builder().with_upgrade(true).build();
But i never receive any query on my http_request_update. It was working well previously.
Edit : i get the CertificateOutdated error.
Edit 2: Needed to put let url = pic.auto_progress(); at the closest possible to the queries. Can we change that ?
The range requests are not made by the browser, they are made by the HTTP Gateway. The browser sends a normal GET request and gets back a 200 response with the full response body.
Yes…but Safari doesn’t do this. I safari sees a video file it intercepts the request and sends a range request with a length of 1 to get the whole size and then it starts sending chunked request to get pieces of the video. Basically video files served from the IC are broken on safari unless you use a proxy and preserve range requests. You can lie to safari and it won’t care, ie you can provide a 2MB block when it asks for just the first byte, but you have to provide the range headers that show the length or it just twiddles its thumbs.