Connecting to api.cerotrade.cl failed: Failed to directly connect: error trying to connect: tcp connect error: deadline has elapsed

Good day everyone,
I am trying to deploy my app to the mainnet, connecting to an API (just your average express, nginx, api, here is the repository https://github.com/Cero-Trade/API-CeroTrade).

Everything is working fine locally, I can access the api normally with Postman and I can ping the app with this ipv6 tester:

PING api.cerotrade.cl(2600:1f16:b33:c700:f175:4526:83db:bafb (2600:1f16:b33:c700:f175:4526:83db:bafb)) 56 data bytes
64 bytes from 2600:1f16:b33:c700:f175:4526:83db:bafb (2600:1f16:b33:c700:f175:4526:83db:bafb): icmp_seq=1 ttl=54 time=21.8 ms
64 bytes from 2600:1f16:b33:c700:f175:4526:83db:bafb (2600:1f16:b33:c700:f175:4526:83db:bafb): icmp_seq=2 ttl=54 time=21.1 ms
64 bytes from 2600:1f16:b33:c700:f175:4526:83db:bafb (2600:1f16:b33:c700:f175:4526:83db:bafb): icmp_seq=3 ttl=54 time=21.2 ms

— api.cerotrade.cl ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 21.114/21.370/21.793/0.300 ms

However, when trying to call the register method through the the mainnet I get this error:

rror: Call was rejected:
Request ID: 6cffb78e3b6d7f4d3072e0809d6c5caf1f7725fafd922af05b13a2dfa6e807ad
Reject code: 4
Reject text: Connecting to api.cerotrade.cl failed: Failed to directly connect: error trying to connect: tcp connect error: deadline has elapsed

Does anyone know what could be up?

This error means that the API connection has timed out. Yes, this error is most common when the domain is not IPv6 compatible.

It does not look like Cerotrade.cl is IPv6 compatible. Can you please check again?

I created api in a google cloud because they support IPV6, and from the google cloud I forward to the IPV4 api. Do not forget the idemponetncy when POSTING.

cerotrade.cl is not, api.cerotrade.cl is! according to this webpage atleast: PlainProxies - IPv6 Compatibility Checker

Both the domain and subdomain need to be on IPv6.

1 Like

I’ll try this out, thanks!

Hey, so I am still getting the same error, even though my domain is now IPv6 supported. Any ideas on what else could be up?

I can get back to you. Do you intend to make the repo public? Or do you have the HTTP Outcalls code snippet?

Just made it public, thanks for the heads up! Also our project repo is public GitHub - Cero-Trade/mvp1.0: Primera versión del MVP en Motoko por el Cero Team.

1 Like

Hey Jennifer! Any updates on this? Still having the same problem

It is still showing that cerotrade.cl does not have IPv6 support activated. It looks like the checker that you are using is only checking the presence of AAAA records and not establishing a direct connection.

What is your DNS provider?

Cloudfare. Actually, this other tester shows that the error is not having AAAA records (? very strange.

Hmm, did you purchase the domain directly through Cloudflare or are using a partial setup or a hosting partner?

It sounds like IPv6 is not enabled on the root domain for partial setups and hosting partners. If you are using a hosting partner, you may want to reach out to the hosting partner about full IPv6 compatibility. If you have a partial setup, you should convert to a full setup.

Hi Jennifer,

I am seeking further assistance with a persistent issue regarding IPv6 connectivity for my API subdomain (api.cerotrade.cl). Despite extensive troubleshooting, the issue remains unresolved. Here is a summary of the problem and everything we have tried so far:

Problem Description

  • Issue: Unable to connect to api.cerotrade.cl over IPv6.
  • Error Message: curl: (7) Failed to connect to api.cerotrade.cl port 80: Connection refused

Steps Taken and Checks Performed

Nginx Configuration

  1. Nginx Configuration File:
  • Verified and updated the Nginx configuration to listen on both IPv4 and IPv6 for ports 80 and 443.

nginx

Copiar código

server {
    server_name api.cerotrade.cl;

    location / {
        proxy_pass http://localhost:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_redirect off;
    }

    listen 443 ssl; # managed by Certbot
    listen [::]:443 ssl; # IPv6 listener

    ssl_certificate /etc/letsencrypt/live/api.cerotrade.cl/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/api.cerotrade.cl/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = api.cerotrade.cl) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

    listen 80;
    listen [::]:80; # IPv6 listener

    server_name api.cerotrade.cl;
    return 404; # managed by Certbot
}
  1. Testing and Reloading Configuration:
  • Tested the Nginx configuration for syntax errors (nginx -t) and reloaded Nginx (sudo systemctl reload nginx).
  1. Checking Listening Ports:
  • Used ss -tuln | grep ':80\|:443' to verify that Nginx is listening on the correct ports. However, it only shows Nginx listening on IPv4 addresses.

bash

Copiar código

tcp   LISTEN 0      511    0.0.0.0:80          0.0.0.0:*                   
tcp   LISTEN 0      511    0.0.0.0:443         0.0.0.0:*                   

AWS Security Group Settings

  1. Security Group Rules:
  • Verified and updated AWS security group rules to allow inbound traffic on ports 80 and 443 for both IPv4 and IPv6.Current Inbound Rules:
  • HTTP (IPv4): Port 80, Source 0.0.0.0/0
  • HTTP (IPv6): Port 80, Source ::/0
  • HTTPS (IPv4): Port 443, Source 0.0.0.0/0
  • HTTPS (IPv6): Port 443, Source ::/0

System-Level Firewall and Network Configuration

  1. Firewall Configuration:
  • Ensured UFW is inactive and allowed traffic on ports 80 and 443 for both IPv4 and IPv6.
  • Added ip6tables rules to allow traffic on ports 80 and 443.

bash

Copiar código

sudo ip6tables -A INPUT -p tcp --dport 80 -j ACCEPT
sudo ip6tables -A INPUT -p tcp --dport 443 -j ACCEPT
  1. Network Interface Configuration:
  • Verified the network interface has an IPv6 address and is active using ip -6 addr show.

bash

Copiar código

inet6 2600:1f16:b33:c700:f175:4526:83db:bafb/128 scope global dynamic noprefixroute

Connectivity Testing

  1. Testing IPv6 Connectivity:
  • Used curl -6 http://api.cerotrade.cl to test IPv6 connectivity, resulting in “Connection refused”.
  • Used external tools like IPv6 Ping to check reachability, confirming the IPv6 address is reachable.

Nginx Logs and System Status

  1. Nginx Status and Logs:
  • Checked Nginx status and logs for any errors that might indicate why it is not binding to IPv6 addresses.

bash

Copiar código

sudo systemctl status nginx
sudo tail -n 20 /var/log/nginx/error.log

Domain Setup

In response to your query about the domain setup:

  • Domain Purchase: The domain cerotrade.cl was purchased directly from NIC Chile.
  • DNS Management: The domain is fully managed by Cloudflare with the following nameservers:
    • yadiel.ns.cloudflare.com
    • zoe.ns.cloudflare.com
  • IPv6 Setup: IPv6 is enabled in Cloudflare, and the AAAA records are correctly set up for api.cerotrade.cl.

Conclusion

Despite verifying and updating the Nginx configuration, AWS security group settings, and system-level firewall rules, Nginx is still not listening on IPv6. The instance’s IPv6 address is configured and reachable, yet the server is refusing connections on port 80 over IPv6.

I am looking for further guidance or any additional steps that I might have overlooked to resolve this IPv6 connectivity issue.

Thank you for your assistance!

Thank you for the details. Can you please confirm that this is the error that you are still getting?

Now I am getting a different error: Failed to directly connect: error trying to connect: tcp connect error: Network is unreachable (os error 101)

Yes, this error is still related to IPv6 capability. I will take a closer look at the troubleshooting steps and get back to you by EOD tomorrow PST time.

1 Like

Hello Jennifer, I managed to fix the ipv6 error I believe by changing my nginx config file to the following:

server {
server_name api.cerotrade.cl;

location / {
    proxy_pass http://localhost:3000;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_redirect off;
}

listen 443 ssl; # managed by Certbot
listen [::]:443 ssl; # IPv6 listener

ssl_certificate /etc/letsencrypt/live/api.cerotrade.cl/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/api.cerotrade.cl/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {
if ($host = api.cerotrade.cl) {
return 301 https://$host$request_uri;
} # managed by Certbot

listen 80;
listen [::]:80; # IPv6 listener

server_name api.cerotrade.cl;
return 404; # managed by Certbot

}

Then, i got the correct ports

sudo ss -tuln | grep ‘:80|:443’
tcp LISTEN 0 511 0.0.0.0:80 0.0.0.0:*
tcp LISTEN 0 511 0.0.0.0:443 0.0.0.0:*
tcp LISTEN 0 511 [::]:80 [::]:*
tcp LISTEN 0 511 [::]:443 [::]:*

However, now, I’m getting this error in my dApp:

“Canister http responses were different across replicas, and no consensus was reached”

I believe maybe it is better if I continue on another thread as the problem is solved, I hope anybody with the same error finds this post helpful.

1 Like

Hello Valentina,

Nice, thank you for the update. Glad that you were able to fix it!

This error message means that the responses are different across the nodes.

When you use HTTP Outcalls, all replicas or nodes in the subnet call the API and will return a valid response only if all of the API responses to the nodes are the same.

Do you know why the API responses are different? Is there any part of the response that would be different across different calls?

Yes, I was reading about this in other posts. I’ll test it out tomorrow and get back to you!

1 Like