Using the principals' textual encoding for other things than principals

Developers
1

For principals we use a textual encoding that looks like this: em77e-bvlzu-aq. It is defined in the spec and can encode any blob of data.

There is now the very real possibility that the same encoding gets used for other things than principals. The first such use is going to be for account identifiers from the ICRC-1 standard. But I expect that other uses might soon follow.

Our principal encoding has a popular, similar “sister”: Bitcoin’s bech32. If we look at the use of bech32 then we can see where it is going. Besides Bitcoin addresses, bech32 is used to encode private keys and lightning invoices, and probably other things. If the principal encoding becomes the “bech32 of the IC” then we can expect it to encode principals, account identifiers for various standards (not only ICRC-1), identifiers relevant to certain extensions of ICRC-1, private keys, invoices in payment flows, and other things that we do not foresee.

The question arises whether the encoding should also encode what the encoded blob’s purpose is. By inspecting an encoded string should we be able to tell whether the data is meant to specify a principal, an account identifier, a private key, an invoice, or something else? Or should the encoded data just be a blob and only the context in which it is exchanges defines its use?

If the answer is the former then we have to define how the intended use is encoded. For example, if there is a tag byte in the data then we need to track the used tags in a registry and projects that use the encoding must respect each other’s tags so they don’t overlap.

If we look at precedence from outside the IC then we see that bech32 encodes the type of encoded data in a human-readable prefix. The older Base58 encoding also has tags and version bytes to distinguish Bitcoin addresses and private keys. Ethereum addresses on the other hand are raw hex data without any identifying data pieces.

I am writing this post now because there is a fork in the road ahead. We are approching the fork with the first use of the encoding outside of principals by ICRC-1. The reason that both roads ahead are still open is that principals have a “type byte” which sits in the last byte position of the data and has a value between 01 and 04. So we can decide to
a) define a scheme that, beginning with the last byte, allows one to determine what the meaning of the encoded data is (only the values 01-04 are already taken), or
b) not care and leave the meaning entirely to the context in which the encoding is exchanged.

I think that anyone on the IC, not only people involved in ICRC-1, must be aware of this and share their opinion. In particular, if we go with a) then the scheme must be agreed upon because it only works if all future uses respect the scheme.

5 Likes
2

Companies like Slack and Stripe use a human-readable prefix with an underscore separator for Base64-encoded access tokens.

These are also easy to search for by static analysis tools.

Underscore isn’t a valid Base64 character so there’s no chance of the prefix colliding with something randomly generated.

For the IC, perhaps that could look something like this:

  • No prefix is assumed to be a Principal for backwards compatibility
  • principal_xxxxx-xx to be explicit
  • icrc-1_xxxxx-xx
4 Likes
3

I love the idea of invoices.

4

ICRC-1 wants to use 0xff for the last byte for account identifiers: Ledger & Tokenization Working Group Update - #21 by roman-kashitsyn

5

One concern about textual prefixes/suffixes is that it’s unclear how to include them in the checksum. Bech32 forms the human-readable part into the checksum. I believe if we want to have the human-readable part, we should extend the principal encoding to do the same.

6

That should be possible if there is demand for it. We can just take the pre-fix bytes and put them under the checksum.

However, another way would be to “register” the available human-readable prefixes and map them to tag bytes. The front-end can look at the tag byte and display the corresponding human-readable prefix. Conversely, when parsing an identifier with human-readable prefix the frontend can compare it to the tag byte and reject if they don’t match. The tag byte is under the checksum so this achieves the same outcome.

Downside may be that it’s not as self-contained as a bech32 string because knowledge of the registered human-readable prefixes is required. But that doesn’t seem to be a big deal because code only needs to know about the prefixes that it is meant to process.