Decode canister response to query request

Developers
1

Hello everyone,

I’m a data engineer working on a Python DFINITY (light) agent. I can make successful “query” http requests to public canisters, but am now having issues parsing the response (porting over the response decoding code from the JS DFINITY agent).

I’ve spent quite a bit of time studying the candid spec and feel like I know what is going on, but the response doesn’t quite match up with how I understand the candid spec.

Anyone have experience with the decode function in the DFINITY agent?

Here are the first 300 (or so) bytes in the response I am trying to parse:
DIDL\x0fl\x04\xb2\xce\xef/q\xdd\xd1\x91D\x01\xab\x87\xa2\x8b\x05\x0b\xc7\xeb\xc4\xd0\tqn\x02l\x04\xf1\xc7\xb8\x92\x01x\x9a\x8a\x90\xf9\x02\x03\xcf\xb4\xc7\xd2\x04x\xc9\xe2\xd7\xd6\x04xm\x04l\x14\xdb\xb7\x01x\xef\xd6\xe4\x02q\x8e\xe0\x9c"t\x8b\xc9\xd9+~\x98\xab\xec\x81\x01q\x8d\x84\xe4\xd9\x02~\xb3\xb0\xda\xc3\x03\x05\xa2\xf5\xed\x88\x04q\xe4\xcb\xd8\x8a\x04\x07\xc0\xe5\xeb\x9b\x05q\xaa\xac\xd9\xd0\x06t\xad\xe9\xaf\x8d\x07~\x90\x86\xfa\xfa\x07\n\xe9\x90\xcc\x8a\x08q\xa7\xf7\xda\xfd\x08h\x8f\xbd\xd6\xb4\x0bx\xa7\xc9\x9f\xef\x0bt\xf8\xbe\xf3\xf7\r~\xd1\x89\xe2\xa7\x0f~\xac\xf5\x95\xe9\x0f\tl\x07\xdb\xb7\x01h\xee\x98\xfd\xb5\x01x\x96\x8c\xae\x87\x02q\x97\xaa\x99\xf1\x04x\xaa\xac\xd9\xd0\x06t\xe9\x90\xcc\x8a\x08q\x97\x81\xd3\xad\x0b\x06l\x02\x8f\xc1\xd4\xfb\x06h\x8c\xf7\xf1\x9d\t~n\x08l\x05\xdb\xb7\x01x\x98\xab\xec\x81\x01q\xb3\xb0\xda\xc3\x03\x05\xa7\xf7\xda\xfd\x08h\xac\xf5\x95\xe9\x0f\tl\x05\xdb\xb7\x01x\x8b\xc9\xd9+~\xcb\xe4\xfd\xc7\x04q\xab\xc6\xd7\xe2\x04q\xe9\x90\xcc\x8a\x08qnxn\x0cm\rl\x02\xba\xa9\xce\x04q\xab\x87\xa2\x8b\x05\x0emq\x01\x00\x05happy\x01\n\x00\x00\x00\x00\x00\x00\x00\nZ\x91\x00\x00\x00\x00\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x05DSCVR\x00\x01\x1d\xcc\x0b\xb9}zG\xf6\xbbf_\xe6\x11v\\\x9b)$\xd8\xf0N\xefH\x1d\xaf96\xbe\x02\x00\x00\x00\x00\x00\x00\x00\x00\x08Rhiannon\xe8\x91\xca\x0b\x00\x00\x00\x00\x00\xbf\x85\xa2r\x95\x82\x16\x00\x01\x1d\xcc\x0b\xb9}zG\xf6\xbbf_\xe6\x11v\\x9b)$\xd8\xf0N\xefH\x1d\xaf96\xbe\x02\x01j

I’m specifically having trouble parsing the type definition table–more specifically a record type within the type definition table.

Thank you!

2

Replacing \x with \, you can feed the message into didc for decoding, e.g. didc decode -f blob 'blob "DIDL\0fl\04\b2\ce...."'

Here is the type table I get from didc:

type table0 = record {
         100_394_802 : text;
         142_895_325 : table1;
         1_365_803_947 : table11;
         2_584_819_143 : text;
       }
       type table1 = opt table2
       type table10 = opt nat64
       type table11 = opt table12
       type table12 = vec table13
       type table13 = record { 9_671_866 : text; 1_365_803_947 : table14 }
       type table14 = vec text
       type table2 = record {
         307_110_897 : nat64;
         790_889_754 : table3;
         1_246_878_287 : nat64;
         1_255_534_921 : nat64;
       }
       type table3 = vec table4
       type table4 = record {
         23_515 : nat64;
         5_843_823 : text;
         71_774_222 : int64;
         91_645_067 : bool;
         272_307_608 : text;
         725_156_365 : bool;
         947_296_307 : table5;
         1_092_319_906 : text;
         1_096_164_836 : table7;
         1_400_566_464 : text;
         1_779_848_746 : int64;
         1_907_094_701 : bool;
         2_136_900_368 : table10;
         2_169_702_505 : text;
         2_411_117_479 : principal;
         3_063_258_767 : nat64;
         3_186_091_175 : int64;
         3_741_114_232 : bool;
         4_109_927_633 : bool;
         4_247_091_884 : table9;
       }
       type table5 = record {
         23_515 : principal;
         381_635_694 : nat64;
         552_306_198 : text;
         1_311_134_999 : nat64;
         1_779_848_746 : int64;
         2_169_702_505 : text;
         3_048_521_879 : table6;
       }
       type table6 = record { 1_869_947_023 : principal; 2_478_603_148 : bool }
       type table7 = opt table8
       type table8 = record {
         23_515 : nat64;
         272_307_608 : text;
         947_296_307 : table5;
         2_411_117_479 : principal;
         4_247_091_884 : table9;
       }
       type table9 = record {
         23_515 : nat64;
         91_645_067 : bool;
         1_224_700_491 : text;
         1_280_697_131 : text;
         2_169_702_505 : text;
       }
4 Likes
3

Wow, nice! This will be extremely helpful in getting things going. Thank you!

4

@chenyan thanks again for the help. I am now correctly parsing the type table, and have started iterating through the type table to parse the response values. :tada: I’m almost there. :slight_smile:

I’ve found candid documentation for (and have successfully implemented) parsing text, nat, bool, records, int, etc, but I don’t understand what table1 means in the first table0 record: 142_895_325 : table1;

What type is table1? Opt? I couldn’t find whether I should be looking for memory/length in the first byte or whether I should be pulling a set number of bytes.

For reference, table1 and table11 both fit into this byte sequence (I know because there is text before and after so I could isolate the table bytes): \x01\n\x00\x00\x00\x00\x00\x00\x00\n\xce\x96\x00\x00\x00\x00\x00\x00\x00\x06\x00\x00\x00\x00\x00\x00\x00\x00

Thank you!

5

table1 means the second entry in the type table.

type table1 = opt table2
type table2 = record {
         307_110_897 : nat64;
         790_889_754 : table3;
         1_246_878_287 : nat64;
         1_255_534_921 : nat64;
       }

So 142_895_325 : table1 means opt record { 307_110_897 : nat64; ... }.

2 Likes
6

Ah… this makes sense. And makes for some recursive table fun! This was the last missing piece for me I think, so should have something working soon.

1 Like