Title: Error 503 When Deploying Canister – Asset Sync Failure

Developers CDK
1

Question:

Hi everyone,
I’m encountering an issue when trying to deploy my canister on the Internet Computer. The deployment fails with a 503 Service Unavailable error during the asset synchronization process.

Here’s the error message I get when running dfx deploy metabuildcity_frontend --network ic:

Error: Failed while trying to deploy canisters.
Caused by: Failed while trying to install all canisters.
Caused by: Failed to install wasm module to canister 'metabuildcity_frontend'.
Caused by: Failed to store assets in canister 'metabuildcity_frontend'.
Caused by: Failed asset sync with canister azhuu-hqaaa-aaaam-aeggq-cai.
Caused by: Failed to create project asset: Failed to create encoding: Failed to create chunk: Failed to create chunk: The replica returned an HTTP Error: Http Error: status 503 Service Unavailable, content type "text/plain; charset=utf-8", content: error: upstream_error
details: The HTTP gateway is temporarily unable to process the request. Please try again later.
Caused by: Failed to create encoding: Failed to create chunk: Failed to create chunk: The replica returned an HTTP Error: Http Error: status 503 Service Unavailable, content type "text/plain; charset=utf-8", content: error: upstream_error
details: The HTTP gateway is temporarily unable to process the request. Please try again later.
Caused by: Failed to create chunk: Failed to create chunk: The replica returned an HTTP Error: Http Error: status 503 Service Unavailable, content type "text/plain; charset=utf-8", content: error: upstream_error
details: The HTTP gateway is temporarily unable to process the request. Please try again later.

Context & Steps Taken:

  • Project Setup:
    • I’m deploying a React-based frontend with Unity WebGL assets stored in the canister.
    • My canister is running fine, and I verified it has sufficient cycles (1,464,511,352,617 Cycles).
    • The memory allocation and compute allocation are both at 0.
  • What I Tried:
    1. Checked the ICP Network Status – No major issues reported at the time.
    2. Synced assets manually:
dfx canister --network ic asset sync metabuildcity_frontend

But it still fails at chunk creation.
3. Cleared and reuploaded assets:

dfx canister --network ic asset clear metabuildcity_frontend
dfx canister --network ic asset sync metabuildcity_frontend
  1. Increased canister memory allocation:
dfx canister --network ic update-settings metabuildcity_frontend --memory-allocation 3221225472
  1. Attempted deployment with verbose logs:
dfx deploy metabuildcity_frontend --network ic --verbose

The error persists.

Questions for the Community:

  1. Is there a known issue with asset synchronization on ICP that could be causing this?
  2. Are there any additional settings I should check to ensure my assets don’t exceed size limits?
  3. Would splitting assets into even smaller chunks help? If so, what’s the recommended approach?
  4. Any other suggestions or debugging steps that might help resolve this issue?

Thanks in advance for your help! :rocket:

2

The only solution I can offer for 503 is “try again” :slightly_frowning_face: It is possible that you tried to deploy when the subnet did an upgrade. That is an event that is likely to produce some 503 errors. Maybe @rbirkner can say more?

3

Hey @herbertzap

The 503 upstream_error is returned for many different reasons. It is however not due to a subnet upgrade. In that case, you would get a 503 no_healthy_nodes.

It would help for us to look into it if you could provide us with the canister ID and the rough time around which you tried to deploy.