The ICRC-3 standard specifies that the hash tree in the certificate returned by icrc3_get_tip_certificate
Should have the following leaves: last_block_index (ULEB128 encoded) last_block_hash
Unfortunately, you are correct. When the standard was being drafted, the ledgers were already providing certificates of a different form than those in the standard. One relevant piece of information is that IC certificates can be extended with new pieces of data, provided that these pieces of data sit at different places in the certificate tree than those that already exist. So, if these paths were different we could just add the paths, as specified in the ICRC-3 standard (and with the encoding specified in the standared). So we could add last_block_hash to the certificate, but we have an overlap with last_block_index.
As far as I can tell there are two paths forward:
change the format of the certificates to conform to the standard. The difficulty here is that this change is not backwards compatible so we need to also deal with clients that already use it.
change the standard and use a different name for the path to last_block_index, so we can provide certificates that do not break the clients & confirm with the standard. Changing the standard is most likely not easy due to procedural difficulties, although there’s a good case here that should help trump these difficulties. Also, in principle this may break other potential implementations which do follow the standard (I am not aware of any)
I think its fine to do either. From what I can see, its not widely used (0 results on github for clients calling icrc3_get_tip_certificate). @skilesare 's implementation would break though (if 2. is how we proceed): GitHub - PanIndustrial-Org/icrc3.mo: ICRC3 in motoko
Its not clear which is more common as there’s not much public code: People verifying certificates for SNS ledgers OR People verifying certificates for properly implemented ICRC-3 Tokens
A non breaking update to standard would look something like this:
Check if tip_hash leaf is present. If yes: last_block_index is a 64 bit unsigned integer, Big Endian Encoded
Else, last_block_hash MUST be present last_block_index is a uleb128 encoded integer
The certificate that is returned by the ledger needs to have the paths that are already there (with the data encoded as is) because there may be clients of the ledger that already make use of this certificate. If we could get the clients to update (along the lines you suggested) we could transition to an implementation that obeys the specification.
One set of clients that we know of is are the Rosetta ICRC nodes, but those are not under our control and it may be difficult to know if/when they have been udpated.
So, IMO the simpler path forward is to modify the standard to say that the certificate holds data at, say, icrc-3_last_block_index and icrc-3_last_block_hash, since afaict the current standard is not being leveraged by clients of the ICRC-3 ledgers since the mismatch that you identified (thanks a lot!, btw) would have already surfaced.
The question is if there are some other implementations of the ledger that closely follow the standard, since then we have to find a path forward that deals with that issue as well.
Ah, I see what you mean: yes, that change to the standard would also deal with the existence of other ledgers that implement the current standard appropriately.
Yeah, to be more specific the standard would have to be updated to the following:
Blocks Verification
The Ledger MUST certify the last block (tip) recorded. The Ledger MUST allow to download the certificate via the icrc3_get_tip_certificate endpoint. The certificate follows the IC Specification for Certificates. The certificate is comprised of a tree containing the certified data and the signature. The tree MUST contain two leaves
last_block_index: the index of the last block in the chain. The values must be expressed as leb128 OR as a 64 Bit unsigned integer, Big Endian Encoded
a leaf with the hash of the last block in the chain. This leaf can must be called last_block_hash if last_block_index is unsigned leb128 encoded. This leaf must be called tip_hash if last_block_index is unsigned 64 bit integer, big endian encoded
implementors may choose last_block_hash or tip_hash based on the encoding of last_block_index. The tree MUST NOT contain both of these together
Clients SHOULD download the tip certificate first and then download the block backward starting from last_block_index and validate the blocks in the process.
Identification of the encoding of last_block_index must be done in the following manner:
If tip_hash leaf is present in the tree, last_block_index is unsigned 64 bit big endian encoded integer, else if last_block_hash leaf is present in the tree, last_block_index is unsigned leb128 encoded
Validation of block i is done by checking the block hash against
if i + 1 < len(chain) then the parent hash phash of the block i+1
otherwise the last_block_hash (or tip_hash) in the tip certificate.