Rosetta API Updates | Neuron Custody

mariop · May 15, 2023, 4:40pm

I just tried with both versions and both worked for me. I don’t know what’s going on but I know that the latest version is being used by various companies atm.

plsak · May 15, 2023, 5:16pm

Thank you, as I got the older version working it should be enough for me to poke around when have a time - if it’s working for others then it’s not so important anyways (I personally suspect some change/problem with ic_ledger_canister_blocks_synchronizer::canister_access - Fetching the root key from the replica because it was not set).

FYI this is the error response when 8080 is already in use (reported by docker as it does the OS bind) - the error I’m getting seems to be related to internal rust execution - that’s why I included the backtrace:

docker: Error response from daemon: driver failed programming external connectivity on endpoint great_shaw (0f527c4c4281216ef62475b82fff9737d2cbd2fac950eeae25d4a074b6eb180d): Bind for 0.0.0.0:8080 failed: port is already allocated.

Plus, actually my main interest is if I can ‘import’ my II created neurons for management via the REST API (didn’t have time to go though the documentation yet).

if yes, I’d also suggest to give the container secondary purpose - include bash, curl, basic CLI utils and prepare functions to give users option to:
- start container as a daemon a connect to it via docker exec ... bash
- use simple CLI commands for personalized configuration and to access all the API functionality - to manage own neurons (as mentioned, not sure if that is currently possible)
- I could eventually even help with this as have done it for my REST API server in one of previous jobs - and it was very very useful

Re-testing of the older/latest version (on my mac):

$ docker images | grep dfinity
dfinity/rosetta-api                                          latest     2e50562006e8   3 months ago    56MB
dfinity/rosetta-api                                          20221018   37ebf03533c3   6 months ago    53.8MB

$ docker run --interactive --tty --publish 8080:8080 --rm dfinity/rosetta-api:latest
16:43:46.724998 INFO [main] ic_rosetta_api - Starting ic-rosetta-api, pkg_version: 1.8.0
16:43:46.725042 INFO [main] ic_rosetta_api - Listening on 0.0.0.0:8080
16:43:46.725052 WARN [main] ic_rosetta_api - Data certificate will not be verified due to missing root key
16:43:46.725076 INFO [main] ic_rosetta_api - Token symbol set to ICP
16:43:46.725358 WARN [main] ic_ledger_canister_blocks_synchronizer::canister_access - Fetching the root key from the replica because it was not set
thread 'main' panicked at 'Failed to initialize ledger client: InternalError(false, Details { error_message: Some("An error happened during communication with the replica: error sending request for url (https://exchanges.testnet.dfinity.network/api/v2/status): error trying to connect: tcp connect error: Cannot assign requested address (os error 99)"), extra_fields: {} })', rs/rosetta-api/src/main.rs:183:35
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

$ docker run --interactive --tty --publish 8080:8080 --rm dfinity/rosetta-api:20221018
16:44:04.731286 INFO [main] ic_rosetta_api - Starting ic-rosetta-api, pkg_version: 1.7.2
16:44:04.731411 INFO [main] ic_rosetta_api - Listening on 0.0.0.0:8080
16:44:04.731430 WARN [main] ic_rosetta_api - Data certificate will not be verified due to missing root key
16:44:04.731473 INFO [main] ic_rosetta_api - Token symbol set to ICP
16:44:05.061400 INFO [main] ic_ledger_canister_blocks_synchronizer::ledger_blocks_sync - Loading blocks from store
16:44:05.061480 INFO [main] ic_ledger_canister_blocks_synchronizer::ledger_blocks_sync - Ledger client is up. Loaded 0 blocks from store. First block at None, last at None
16:44:05.061579 INFO [main] ic_rosetta_api - Network id: NetworkIdentifier { blockchain: "Internet Computer", network: "00000000000000020101", sub_network_identifier: None }
16:44:05.061949 INFO [main] ic_rosetta_api::rosetta_server - Starting Rosetta API server
16:44:05.184503 INFO [main] ic_ledger_canister_blocks_synchronizer::ledger_blocks_sync - Syncing 1107 blocks. New tip will be 1106
16:44:05.446169 INFO [main] ic_ledger_canister_blocks_synchronizer::ledger_blocks_sync - You are all caught up to block 1106
...

`RUST_BACKTRACE=full`:

16:59:55.323244 INFO [main] ic_rosetta_api - Starting ic-rosetta-api, pkg_version: 1.8.0
16:59:55.323295 INFO [main] ic_rosetta_api - Listening on 0.0.0.0:8080
16:59:55.323337 WARN [main] ic_rosetta_api - Data certificate will not be verified due to missing root key
16:59:55.323415 INFO [main] ic_rosetta_api - Token symbol set to ICP
16:59:55.323581 WARN [main] ic_ledger_canister_blocks_synchronizer::canister_access - Fetching the root key from the replica because it was not set
thread 'main' panicked at 'Failed to initialize ledger client: InternalError(false, Details { error_message: Some("An error happened during communication with the replica: error sending request for url (https://exchanges.testnet.dfinity.network/api/v2/status): error trying to connect: tcp connect error: Cannot assign requested address (os error 99)"), extra_fields: {} })', rs/rosetta-api/src/main.rs:183:35
stack backtrace:
   0:     0x55a069463c50 - std::backtrace_rs::backtrace::libunwind::trace::h32eb3e08e874dd27
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x55a069463c50 - std::backtrace_rs::backtrace::trace_unsynchronized::haa3f451d27bc11a5
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55a069463c50 - std::sys_common::backtrace::_print_fmt::h5b94a01bb4289bb5
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys_common/backtrace.rs:66:5
   3:     0x55a069463c50 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hb070b7fa7e3175df
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys_common/backtrace.rs:45:22
   4:     0x55a06948950e - core::fmt::write::hd5207aebbb9a86e9
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/fmt/mod.rs:1202:17
   5:     0x55a06945cec5 - std::io::Write::write_fmt::h3bd699bbd129ab8a
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/io/mod.rs:1679:15
   6:     0x55a069465453 - std::sys_common::backtrace::_print::h7a21be552fdf58da
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys_common/backtrace.rs:48:5
   7:     0x55a069465453 - std::sys_common::backtrace::print::ha85c41fe4dd80b13
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys_common/backtrace.rs:35:9
   8:     0x55a069465453 - std::panicking::default_hook::{{closure}}::h04cca40023d0eeca
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:295:22
   9:     0x55a06946513f - std::panicking::default_hook::haa3ca8c310ed5402
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:314:9
  10:     0x55a069465b5a - std::panicking::rust_panic_with_hook::h7b190ce1a948faac
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:698:17
  11:     0x55a069465a57 - std::panicking::begin_panic_handler::{{closure}}::hbafbfdc3e1b97f68
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:588:13
  12:     0x55a0694640fc - std::sys_common::backtrace::__rust_end_short_backtrace::hda93e5fef243b4c0
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys_common/backtrace.rs:138:18
  13:     0x55a069465772 - rust_begin_unwind
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:584:5
  14:     0x55a069486bd3 - core::panicking::panic_fmt::h8d17ca1073d9a733
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/panicking.rs:142:14
  15:     0x55a06877130e - ic_rosetta_api::main::{{closure}}::h799450387c351464
  16:     0x55a0687e9d5d - std::thread::local::LocalKey<T>::with::h048ad4042cc1c2ae
  17:     0x55a06876c1f8 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::ha7ea65f8031aec79
  18:     0x55a06879b060 - tokio::runtime::scheduler::current_thread::Context::enter::hd49f70e033169598
  19:     0x55a068807ff1 - tokio::macros::scoped_tls::ScopedKey<T>::set::hf2a10f0988424376
  20:     0x55a06879ac8f - tokio::runtime::runtime::Runtime::block_on::hc3b55ecb8eabc9ae
  21:     0x55a068784285 - ic_rosetta_api::main::h7426bf004f709972
  22:     0x55a068777b13 - std::sys_common::backtrace::__rust_begin_short_backtrace::h930e0d924b1945e7
  23:     0x55a068790439 - std::rt::lang_start::{{closure}}::hd61fb70ac743a4c4
  24:     0x55a0694578bf - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::hb69be6e0857c6cfb
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:283:13
  25:     0x55a0694578bf - std::panicking::try::do_call::h396dfc441ee9c786
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
  26:     0x55a0694578bf - std::panicking::try::h6cdda972d28b3a4f
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
  27:     0x55a0694578bf - std::panic::catch_unwind::h376039ec264e8ef9
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
  28:     0x55a0694578bf - std::rt::lang_start_internal::{{closure}}::hc94720ca3d4cb727
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:148:48
  29:     0x55a0694578bf - std::panicking::try::do_call::h2422fb95933fa2d5
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:492:40
  30:     0x55a0694578bf - std::panicking::try::h488286b5ec8333ff
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:456:19
  31:     0x55a0694578bf - std::panic::catch_unwind::h81636549836d2a25
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panic.rs:137:14
  32:     0x55a0694578bf - std::rt::lang_start_internal::h6ba1bb743c1e9df9
                               at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/rt.rs:148:20
  33:     0x55a0687842e8 - main
  34:     0x7f5b3fa7cd0a - __libc_start_main
  35:     0x55a0687316ee - _start
  36:                0x0 - <unknown>

ICdex · January 24, 2024, 3:12pm

Have you made any progress? When will the ICRC-1 token be launched on the exchange?please

plsak · January 28, 2024, 10:22am

FYI @mariop, so it seems that more people (still) have a problem with the latest image and it might be worth doing more investigation:

mariop · January 29, 2024, 9:16am

I see. Maybe the issue is the connection from your machine to the testnet. Can you try to run the following command:

$ docker run --interactive --tty --publish 8080:8080 --rm dfinity/rosetta-api:v2.0.0 --mainnet

plsak · January 29, 2024, 10:45am

Unfortunately I might not be able to do here more testing for personal/business reasons anytime soon, but possibly @vafi (who reported the last problem) might be able to help?

vafi · January 29, 2024, 10:56am

the old image produces problems for me as well, different problems. The dev team will need to revise the rosetta docker image.

Not to backseat drive, I can help @dfinity

vafi · January 29, 2024, 10:59am

Hi, @plsak and I were having a similar experience trying to connect to testnet from our local machines and remote server. The command you provided works for me and I am able to run successfully when connecting to mainnet, there appears to be an issue with the testnet.

plsak · January 29, 2024, 11:41am

Finally also got access to one convenient (linux) environment, following are results of the tests - not sure about network of the older version (starts fine, based on network: "00000000000000020101" looks like mainnet) but for the new one looks like problem with the testnet:

$ docker images | grep dfinity
dfinity/rosetta-api   latest           06f73feccee9   11 days ago     54.8MB
dfinity/rosetta-api   v2.0.0           1c939589db70   11 days ago     54.8MB
dfinity/rosetta-api   20221018         37ebf03533c3   15 months ago   53.8MB

`v2.0.0`

$ docker run --interactive --tty --publish 8080:8080 --rm dfinity/rosetta-api:v2.0.0
2024-01-29T11:23:42.452464Z  INFO rs/rosetta-api/src/main.rs:141: Starting ic-rosetta-api, pkg_version: 2.0.0
2024-01-29T11:23:42.452507Z  INFO rs/rosetta-api/src/main.rs:147: Listening on 0.0.0.0:8081
2024-01-29T11:23:42.452546Z  INFO rs/rosetta-api/src/main.rs:150: Internet Computer URL set to https://exchanges.testnet.dfinity.network/
2024-01-29T11:23:42.452559Z  WARN rs/rosetta-api/src/main.rs:182: Data certificate will not be verified due to missing root key
2024-01-29T11:23:42.452576Z  INFO rs/rosetta-api/src/main.rs:207: Token symbol set to ICP
2024-01-29T11:23:42.452796Z  WARN rs/rosetta-api/ledger_canister_blocks_synchronizer/src/canister_access.rs:71: Fetching the root key from the replica because it was not set
thread 'main' panicked at rs/rosetta-api/src/main.rs:248:35:
Failed to initialize ledger client: InternalError(false, Details { error_message: Some("An error happened during communication with the replica: error sending request for url (https://exchanges.testnet.dfinity.network/api/v2/status): error trying to connect: tcp connect error: Cannot assign requested address (os error 99)"), extra_fields: {} })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Internet Computer URL set to https://exchanges.testnet.dfinity.network/

$ docker run --interactive --tty --publish 8080:8080 --rm dfinity/rosetta-api:v2.0.0 --mainnet
2024-01-29T11:25:29.203666Z  INFO rs/rosetta-api/src/main.rs:141: Starting ic-rosetta-api, pkg_version: 2.0.0
2024-01-29T11:25:29.203706Z  INFO rs/rosetta-api/src/main.rs:147: Listening on 0.0.0.0:8081
2024-01-29T11:25:29.203745Z  INFO rs/rosetta-api/src/main.rs:150: Internet Computer URL set to https://ic0.app/
2024-01-29T11:25:29.203793Z  INFO rs/rosetta-api/src/main.rs:207: Token symbol set to ICP
2024-01-29T11:25:29.794386Z  INFO rs/rosetta-api/ledger_canister_blocks_synchronizer/src/ledger_blocks_sync.rs:87: Loading blocks from store
2024-01-29T11:25:29.794529Z  INFO rs/rosetta-api/ledger_canister_blocks_synchronizer/src/ledger_blocks_sync.rs:98: Ledger client is up. Loaded 0 blocks from store. First block at None, last at None
2024-01-29T11:25:29.794610Z  INFO rs/rosetta-api/src/main.rs:253: Network id: NetworkIdentifier(NetworkIdentifier { blockchain: "Internet Computer", network: "00000000000000020101", sub_network_identifier: None })
2024-01-29T11:25:29.795086Z  INFO rs/rosetta-api/src/rosetta_server.rs:425: Starting Rosetta API server
2024-01-29T11:25:29.839957Z  INFO rs/rosetta-api/ledger_canister_blocks_synchronizer/src/ledger_blocks_sync.rs:339: Syncing 10100649 blocks. New tip will be 10100648

Internet Computer URL set to https://ic0.app/
blockchain: "Internet Computer", network: "00000000000000020101"

`20221018`

$ docker run --interactive --tty --publish 8080:8080 --rm dfinity/rosetta-api:20221018
11:31:41.215627 INFO [main] ic_rosetta_api - Starting ic-rosetta-api, pkg_version: 1.7.2
11:31:41.215660 INFO [main] ic_rosetta_api - Listening on 0.0.0.0:8080
11:31:41.215682 WARN [main] ic_rosetta_api - Data certificate will not be verified due to missing root key
11:31:41.215720 INFO [main] ic_rosetta_api - Token symbol set to ICP
11:31:41.685269 INFO [main] ic_ledger_canister_blocks_synchronizer::ledger_blocks_sync - Loading blocks from store
11:31:41.685406 INFO [main] ic_ledger_canister_blocks_synchronizer::ledger_blocks_sync - Ledger client is up. Loaded 0 blocks from store. First block at None, last at None
11:31:41.685495 INFO [main] ic_rosetta_api - Network id: NetworkIdentifier { blockchain: "Internet Computer", network: "00000000000000020101", sub_network_identifier: None }
11:31:41.686181 INFO [main] ic_rosetta_api::rosetta_server - Starting Rosetta API server
11:31:42.052324 INFO [main] ic_ledger_canister_blocks_synchronizer::ledger_blocks_sync - You are all caught up to block 138

blockchain: "Internet Computer", network: "00000000000000020101"

plsak · January 29, 2024, 12:40pm

@mariop thanks for the tip, so if the problem is with connection to testnet there is probably no more investigation needed

Now looking at the recent major version bump (2.0.0) this could mean a large functionality upgrade, could you please share more details or a link to the release notes (I didn’t find more info on Docker Hub)?

mariop · January 31, 2024, 9:23am

The CHANGELOG can be found at ic/rs/rosetta-api/CHANGELOG.md at 29567e4ad17f91a0f65259e372a5237b6430acd3 · dfinity/ic · GitHub but the 2.0.0 section is not ready yet. The changes are the following:

BREAKING: new logging library that replaces the old one. Log configuration file is deprecated
FIX: Prohibit Rosetta from spamming the ledger in case of errors at the ledger client
Add timestamp to the blocks table
Add support for list_neurons. Let’s a user query a list of all they neurons a user has created.
Add support for list_known_neurons. Let’s a user query a list of all publicly known neurons.

Sawyer · February 15, 2024, 4:18am

When does rosseta api come for cex listing of icp tokens?

mariop · February 15, 2024, 9:17am

Q1, we are working on it right now.

daijianlin · February 29, 2024, 11:12am

Hey is there any update? any detailed schedule? the last month of Q1 is coming.

mariop · February 29, 2024, 7:29pm

We aim for end of Q1, but that we are working on improving certain bottlenecks which may result in a delay. A preliminary alpha version could be made available in the next weeks.

gatsby_esp · March 1, 2024, 1:21am

So there’s no CEXs for ICP Defi during this bullrun. Amazing.
Can’t believe dfinity let this happen.

daijianlin · March 1, 2024, 5:20am

Thanks for the reply, I appreciate that. Hope you could speed up the work, without the function then all icpcoins could not be listed on cex is ridiculous.

daijianlin · March 6, 2024, 1:29pm

Have a look at this, I believe you need to speed up the process of development.

Topic		Replies	Views
Funding proposal for IC bounties and major contributions to the ecosystem Governance	15	1015	October 27, 2022
Ledger integration for neurons created using Internet Identity General	22	2012	May 11, 2022
Tokenomics Proposal [Community Consideration] Roadmap community-consideration	158	13628	October 20, 2022
Warning - Boom DAO Red Flag! Governance nns , sns	111	4315	October 30, 2023
Staking problems within NNS app Developers	83	6680	February 23, 2024

Rosetta API Updates | Neuron Custody

Re-testing of the older/latest version (on my mac):

RUST_BACKTRACE=full:

v2.0.0

20221018

Related Topics

`RUST_BACKTRACE=full`:

`v2.0.0`

`20221018`