Chain fusion: error -32010 "AlreadyKnown" when sending tx

Dear community,

I use the chainfusion technology to develop ReTransICP. Part of this task it to send a transaction every now and then. This returns an error:

HttpOutcallError(IcError { code: SysTransient, message: "Connecting to rpc.gnosischain.com failed: Failed to directly connect: client error (Connect)" })

“AlreadyKnown” means: the rpc provider has this exact transaction in the mempool and informs me that I can stop sending it.
When using a dedicated rpc I can confirm that the transaction gets sent 28 times (from 28 ICP nodes), and leads to this error 27 times. In other words: the rpc is well behaved.
But the ic-evm-utils trap on every error being returned. Here is the code I found:

match evm_rpc
        .eth_send_raw_transaction(rpc_services, None, tx, cycles)
        .await
    {
        Ok((res,)) => match res {
            MultiSendRawTransactionResult::Consistent(status) => match status {
                SendRawTransactionResult::Ok(status) => status,
                SendRawTransactionResult::Err(e) => {
                    ic_cdk::trap(format!("Error: {:?}", e).as_str());
                }
            },
            MultiSendRawTransactionResult::Inconsistent(_) => {
                ic_cdk::trap("Status is inconsistent");
            }
        },
        Err(e) => ic_cdk::trap(format!("Error: {:?}", e).as_str()),
    }

Since this is not in my project code, I can’t change the behavior. My questions are:

  1. This is ic-evm-utils 0.1.0. I just found there is v1.0.0 now. Is this behavior fixed? How do I update? I think I updated to ic-evm-utils v1.0.0. The relevant code seems to be the same and it behaves the same, too.
  2. What other options do I have? Can I use different methods to send the transaction? Can I catch traps or something like this?
  3. Is anyone else experiencing this? Did you find a solution?
  4. @cryptoschindler have you seen this before? Any ideas?

My code is on github if you are interested, see link above.

This happens on Gnosis chain with these endpoints

All other endpoints I tried give me worse results: they even block reads with HttpOutcallError(IcError { code: SysTransient, message: "Connecting to 1rpc.io failed: Failed to directly connect: client error (Connect)" })

2 Likes

Hey @malteish , first of all thank you for taking the time to write this feedback.
You are correct in that the behaviour of ic-evm-utils is not ideal and in scenarios like yours the error should be handled instead of trapping.

There is an issue for it in the repo, but unfortunately I haven’t had the time yet to implement the necessary changes. If you are interested, I’d be more than happy to accept contributions to the crates. The code lives here, and you can find some context in the docs here. Also, there are plans by @kristofer to write a signer/provider for the new alloy libs, but I’m not sure what the ETA for those is.

Other than that, you can always write your own logic to make calls to the EVM RPC canister, inspired by the code that is present in the repo.

1 Like

I am trying to change the ic-evm-utils as you recommended.
For this, I am extending the SendRawTransactionStatus and recompiling evm-rpc-canister-types with

cargo build --release --target wasm32-unknown-unknown --package evm-rpc-canister-types

That works fine.

I then try to use these types in ic-evm-utils and compile with

cargo build --release --target wasm32-unknown-unknown --package ic-evm-utils

This fails like this:

$ cargo build --release --target wasm32-unknown-unknown --package ic-evm-utils
  Compiling getrandom v0.2.15
  Compiling hex v0.4.3
  Compiling num-traits v0.2.19
  Compiling futures-sink v0.3.30
error: the wasm*-unknown-unknown targets are not supported by default, you may need to enable the "js" feature. For more information see: https://docs.rs/getrandom/#webassembly-support
  -->[censored]/.cargo/registry/src/index.crates.io-6f17d22bba15001f/getrandom-0.2.15/src/lib.rs:342:9
   |
342 | /         compile_error!("the wasm*-unknown-unknown targets are not supported...
343 | |                         default, you may need to enable the \"js\" feature...
344 | |                         For more information see: \
345 | |                         https://docs.rs/getrandom/#webassembly-support...
   | |________________________________________________________________________^

  Compiling futures-util v0.3.30
error[E0433]: failed to resolve: use of undeclared crate or module `imp`
  --> [censored]/.cargo/registry/src/index.crates.io-6f17d22bba15001f/getrandom-0.2.15/src/lib.rs:398:9
   |
398 |         imp::getrandom_inner(dest)?;
   |         ^^^ use of undeclared crate or module `imp`

For more information about this error, try `rustc --explain E0433`.
error: could not compile `getrandom` (lib) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...

Adding the flag to cargo.toml like recommended solved the issue: getrandom - Rust
Is it safe though? Using bad random numbers has lead to many problems already.

To actually use the local package version I had to change my cargo.toml: this version from the upstream repo does not use the local packages.

ic-evm-utils = "1.0.0"
evm-rpc-canister-types = "1.0.0"

This version does:

ic-evm-utils = { path = "packages/ic-evm-utils" }
evm-rpc-canister-types = { path = "packages/evm-rpc-canister-types" }

Unfortunately, now, I can’t update my canister anymore. Running the deploy script fails with this error:

Error: Failed to install wasm module to canister 'chain_fusion'.
Caused by: Failed during wasm installation call
Caused by: The replica returned a rejection error: reject code CanisterError, reject message Error from Canister h4e2i-uaaaa-aaaag-albyq-cai: Canister's Wasm module is not valid: Wasm module has an invalid import section. Module imports function '__wbindgen_describe' from '__wbindgen_placeholder__' that is not exported by the runtime..
This is likely an error with the compiler/CDK toolchain being used to build the canister. Please report the error to IC devs on the forum: https://forum.dfinity.org and include which language/CDK was used to create the canister., error code None

Here is the PR that shows what I changed. It might even be related to the getrandom-thing mentioned in my last post.

Does anyone have any recommendations?

UPDATE: thanks to this post I changed my import of getrandom to “custom”, which fixed the issue:

getrandom = { features = ["custom"] }
1 Like

By now I added code to intercept the AlreadyKnown error, but I can still not change the behavior of the canister. In my last attempt I simply changed the string in the error message I thought was at the root of the issue, but the new message is never used by the canister.

Expected error message: ...Error with my personal message:
Actual error message: [TRAP]: Error: JsonRpcError(JsonRpcError { code: -32010, message: "AlreadyKnown" })

Can someone please help me find which code I need to edit or how to make sure it gets included in the final build?

The build logs suggest the local packages are compiled, so I assumed they are used as well:

   Compiling ic-cdk v0.14.1
   Compiling reqwest v0.12.7
   Compiling ic-cdk-timers v0.7.0
   Compiling evm-rpc-canister-types v1.0.0 (/censored/packages/evm-rpc-canister-types)
   Compiling ic-evm-utils v1.0.0 (/censored/packages/ic-evm-utils)
   Compiling ic-base-types v0.9.0 (https://github.com/dfinity/ic#12156a5b)
   Compiling dfn_core v0.9.0 (https://github.com/dfinity/ic#12156a5b)

Update: found by stupidly replacing all Errors with Error 1 through Error n that I was working on the wrong one. More info soon.

My changes to the ic-evm-utils package were compiled and used, but I tried to catch the error in the wrong place, which is why it didn’t work. Once I identified the proper place, I was able to implement the change I needed: a dedicated status message for transaction AlreadyKnown. Will prepare a PR for the upstream repo.

1 Like

Would it be sufficient for you if we’d just return MultiSendRawTransactionResult and then the result handling is done in the consuming application? or even one level above, just return the call result and don’t do any matching :thinking:
this would take away some of the convenience but gives back control to the consuming party

Here is the PR

You are correctly describing the tradeoffs. Since the point of the package is to provide convenience, I doubt removing this abstraction would generate much value. Instead, I would like to see this approach:

  1. quick fix: instead of trapping, return an error that includes the error string. Then the calling application can treat it.
  2. step by step, add the required status types so the application does not have to do string matching anymore.
2 Likes

Thanks for the PR! Much appreciated. This has been merged here

1 Like

Hey @malteish ,

after a conversation with @gregory-demay I decided to rewrite the library yet again and turn the send_raw_transaction function into a fire and forget. The reasoning behind this was that there is no one-size-fits-all approach in filtering AlreadyKnown messages for all the different RPC providers, as error messages are not standardised for the RPC API.

The idea now is to make a call to send_raw_transaction that either gives you a transaction hash or a system error. In the case of the transaction hash, there’s a high chance that your transaction made it to the RPC. To double check you now call get_transaction_count and check if transaction_count > nonce , using the nonce from the transaction you just submitted. You could then also use eth_getTransactionReceipt which returns null for unknown and pending transactions.

You can see the proposed code changes here