Breaking Registry Changes for Large Records

Developers
1

You can stop reading now if you do not call the following methods of the Registry canister:

  • get_certified_changes_since
  • get_changes_since
  • get_value

Dear most esteemed Registry clients,

TL;DR: DFINITY proposes to make some breaking changes to the aforementioned methods. That is, affected clients will need to make changes, but a gradual migration path is planned.

The problem with the current interface is that messages must not exceed ICP’s 2 MiB size limit. Without these changes, the upcoming vetKeys feature would exceed that limit.

Very briefly, what clients will need to do when fetching a larger record is to download it over multiple calls. Requests would look exactly as they do now. The difference is that clients will need to support a slightly enhanced response. This is only needed if the record is large. If a client is already using a DFINITY-developed Registry library, all they would need to do is upgrade this library.

The exact details of how to migrate are explained in this .proto file, but to slightly elaborate, clients would need to switch from decoding to new “high-capacity” types, which contain an additional large_value_chunk_keys field. Using this field, clients can make calls to a new get_chunk method, thus spreading the large record over multiple calls.

Callers CAN safely switch right now. Such changes will be NEEDED later, when large entries are added to the Registry. It is planned that large entries would be added some time in the second quarter of 2025. The exact date will be announced as the time approaches.

Please, check out the detailed plan in the aforementioned .proto file. If you have concerns, please, reach out. Nobody likes breaking changes, but in order to support larger records, this seems about as minimally invasive and gradual as one can imagine.

As always, your continued support of the Internet Computer Protocol is greatly appreciated. Happy smart contracting!

Sincerely,
Daniel Wong

5 Likes
2

Thanks for the update Daniel. With these changes can you additionally provide candid endpoints for these APIs so they can be more easily consumed by canisters?

3

This x 100 and paginated

4

Thank you for the feedback; it makes sense. It’s not in scope for this project, but maybe we can do it as follow up.

FWIW, while I was designing this, I didn’t see any reason why these must not be available in Candid as well. IOW, we have nothing against what you are asking for. I think it was done this way originally because at the time, Candid was not mature enough.

1 Like
5

A version of the nns_data_provider library that supports chunked responses is available now in the master branch of the ic repo. Therefore, now would be a good time for users of that library to upgrade. This should not require any client code changes (unless of course, I am forgetting something).

The Registry canister has the code needed to start accepting large mutations, but this feature has not been enabled (yet), so there is still time to for clients to migrate (e.g. by simply doing the nns_data_provider library upgrade mentioned in the first paragraph). We would like to enable larger mutations in the Registry canister in a couple of weeks or so. I will update this thread as the exact time becomes clearer.