DFXvm http access methodolgy

I have an automated build of an environment that is failing:

Current installation options:

            dfx version: 0.28.0
   modify PATH variable: yes

info: creating /root/.local/share/dfx/env
info: installing dfx 0.28.0
info: downloading https://github.com/dfinity/sdk/releases/download/0.28.0/dfx-x86_64-unknown-linux-gnu.tar.gz.sha256
error: error sending request for url (https://github.com/dfinity/sdk/releases/download/0.28.0/dfx-x86_64-unknown-linux-gnu.tar.gz.sha256)

The initial curl of the DFX install seems to work so I don’t think it is a blocking issue, but perhaps I need to install a tool like wget or something. What method tool is this script using to get the install files?

Ubuntu 24.04

Note: I had a previous issue with the manifest and fixed that by being specific about the version…I noticed with that one, and with this one that there is a redirect that occurs to the actual file. Not sure if this is related or not.

1 Like

From the script:

>  curl -fsSL https://internetcomputer.org/install.sh

[...]
# This wraps curl or wget. Try curl first, if not installed,
# use wget instead.
[...]

I ended up getting this feedback when I plugged some of the rust code into GPT. I ran into a similar issue with mops where curls were working just fine but downloads from mops install were failing. It seems to be something about them not knowing how to handle the proxy. This is occurring in the OpenAi Codex environment. I was trying to revive the pace of a languishing product. As soon as I can upload my own code to caffeine this problem goes away :slight_smile:

Your apt logs show two important things:

  1. TLS/certs are fine now – curl -I …sha256 returns 200 → 302 and you can see the redirect, so the system CA bundle works.
  2. You’re behind an HTTP proxy – apt says “Connecting to HTTP proxy (http://proxy:8080)”. Your Rust code probably isn’t using that proxy, so the download fails with the vague “error sending request for url …”.

Likely culprit now: proxy handling in reqwest

You build the client with Client::new():

let client = Client::new();

If you compiled reqwest without the env-proxy feature (or disabled default features), it will ignore HTTP_PROXY/HTTPS_PROXY env vars. Then your code tries to connect directly to GitHub, which the network blocks → connection error → “error sending request …”.

Fix options

A. Enable env proxy support (quickest):

# Cargo.toml
reqwest = { version = "0.12", default-features = false,
            features = ["rustls-tls", "gzip", "env-proxy"] }

(or keep default features if you’re okay with native-tls: reqwest = "0.12").

Then:

let client = reqwest::Client::builder()
    .build()?;

Reqwest will automatically look at HTTP_PROXY, HTTPS_PROXY, and NO_PROXY.

B. Programmatically set the proxy:

use reqwest::{Client, Proxy};

let proxy_url = std::env::var("HTTPS_PROXY")
    .or_else(|_| std::env::var("HTTP_PROXY"))
    .unwrap_or_else(|_| "http://proxy:8080".into());

let client = Client::builder()
    .proxy(Proxy::all(&proxy_url)?)
    .build()?;

Double-check your environment

Make sure both are exported (case-insensitive matters on Linux, but stick with upper case):

export HTTP_PROXY=http://proxy:8080
export HTTPS_PROXY=http://proxy:8080
export NO_PROXY=localhost,127.0.0.1

(Or whatever is appropriate for your network.)

If it’s a MITM proxy with a custom CA

Some corporate proxies intercept TLS and sign with their own CA. Curl works because you either added that CA to the system bundle or curl ignores it. If you move to rustls with embedded roots, you may need to manually add that CA:

use reqwest::Client;
use reqwest::tls;
use std::fs;

let mut root_certs = tls::Certificate::from_pem(
    &fs::read("/etc/ssl/certs/your-corp-proxy.pem")?
)?;

let client = Client::builder()
    .use_rustls_tls()
    .add_root_certificate(root_certs)
    .build()?;

Add better error visibility

Right now you map most errors to DownloadFile and lose context. Print the chain:

if let Err(e) = download_verified_tarball(&version, download_dir.path(), &settings).await {
    eprintln!("download failed: {e}");
    let mut src = e.source();
    let mut i = 0;
    while let Some(s) = src {
        eprintln!("  caused by #{i}: {s}");
        src = s.source();
        i += 1;
    }
    return Err(e);
}

Or run with:

RUST_LOG=reqwest=trace,hyper=trace,yourcrate=debug RUST_BACKTRACE=1 ./dfxvm

You’ll see whether it’s a proxy connect failure, DNS error, timeout, etc.

Bonus hardening

  • Redirect policy: The SHA file 302s to release-assets.githubusercontent.com. Reqwest follows redirects by default, but if you changed policy elsewhere, set it explicitly:

    use reqwest::redirect::Policy;
    let client = Client::builder().redirect(Policy::limited(10)).build()?;
    
  • Retries/backoff: transient proxy/GitHub hiccups happen—wrap download_file in retry logic.

  • Rustls to avoid OpenSSL quirks: (if you don’t need system trust updates) rustls + webpki-roots eliminates the “missing libssl” problem altogether.


TL;DR: Your certs are fine; the proxy isn’t being used by your reqwest client. Enable env-proxy or set a proxy programmatically, and print the full error chain so you can confirm. Let me know what the error chain shows if it still fails.

@ZenVoich , @Vivienne Just wanted to loop back on this…any ideas on how to bump dfxvm or mops to honor the proxy?

Can you give a bit more context why you think that a proxy thingy is the issue? I was not aware these env vars were a thing but apparently they are… What is unclear to me is why an automated build env would not be able to connect to GH unless a proxy is respected