# Motoko Serialization **URL:** \ **Category:** Developers\ **Created:** [December 28, 2021, 1:37am UTC](https://forum.dfinity.org/t/motoko-serialization/9954 "2021-12-28T01:37:24Z")\ **Posts on this page:** 16\ **Page:** 2
**Author:** ![jzxchiang](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/jzxchiang/32/2592_2.png) [@jzxchiang](https://forum.dfinity.org/u/jzxchiang)\ **Post date:** [December 30, 2021, 9:23pm UTC](https://forum.dfinity.org/t/motoko-serialization/9954/21 "2021-12-30T21:23:09Z")
> To back up the data in the stable memory I would just take the entire memory pages and ship them off as Blob to another canister. To clarify, do you mean to use the `loadBlob` [function](https://smartcontracts.org/docs/base-libraries/ExperimentalStableMemory.html#loadBlob)? > You should be able to send at least 2 MB in a single inter-canister message. With the stable memory being organized in memory pages of 64kB you can then ship 32 pages at once. Oh interesting, I thought the 2 MB message size limit only applied to [ingress](https://github.com/dfinity/ic/blob/f14eb6e964eea38368bfb423bae00e66c09a2a70/rs/prep/src/subnet_configuration/constants.rs#L23) messages. I didn’t know they also applied to inter-canister messages. ---
**Author:** ![Motokoder](https://avatars.discourse-cdn.com/v4/letter/m/e47c2d/32.png) [@Motokoder](https://forum.dfinity.org/u/Motokoder)\ **Post date:** [December 30, 2021, 11:19pm UTC](https://forum.dfinity.org/t/motoko-serialization/9954/22 "2021-12-30T23:19:05Z")
Thank you @timo for your comprehensive explanation. This gives me a lot to think about. Due to the experimental warning, I have intentionally left out ExperimentalStableMemory from the solution. However, I am very hopeful about leveraging this library in the future. I have a few questions on that topic if you have the time. 1. It sounds like the only way to get a blob from an instance of a complex type is to custom serialize it to a string format like JSON and then call Text.encodeUtf8. Is my understanding correct? 2. If I need to restore all pages of stable memory after an upgrade to some HashMaps, could I still run into the cycle limit? 3. To echo @jzxchiang, does that 2 MB message size limit apply to inter-canister messages? ---
**Author:** ![timo](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/timo/32/5543_2.png) [@timo](https://forum.dfinity.org/u/timo)\ **Post date:** [December 31, 2021, 12:02am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/23 "2021-12-31T00:02:37Z")
> [@Motokoder](#): > > It sounds like the only way to get a blob from an instance of a complex type is to custom serialize it to a string format like JSON and then call Text.encodeUtf8. Certainly not the only way. What complex type are you thinking about? You can probably serialize it directly to binary. > [@Motokoder](#): > > If I need to restore all pages of stable memory after an upgrade to some HashMaps, could I still run into the cycle limit? Theoretically, yes. If that happens then you could spread building the HashMap over several messages. The upgrade could leave the canister in a state in which only a post-upgrade function can be called and no other functions. And then you can call the post-upgrade function multiple times and each time you call it a few more entries get added to the HashMap. There are also ways for a canister to call itself in a loop, then it would be automated. > [@Motokoder](#): > > does that 2 MB message size limit apply to inter-canister messages? As far as I know, yes. Inter-canister messages end up in blocks and there is a block size limit so there must be a message size limit, too. ---
**Author:** ![Motokoder](https://avatars.discourse-cdn.com/v4/letter/m/e47c2d/32.png) [@Motokoder](https://forum.dfinity.org/u/Motokoder)\ **Post date:** [December 31, 2021, 4:02am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/24 "2021-12-31T04:02:40Z")
Thanks again. I like your idea to copy the stable data with multiple operations after the upgrade is finished. I’ll do some testing for the message size limit since the IC code is a bit over my head. > [@timo](#): > > Certainly not the only way. What complex type are you thinking about? You can probably serialize it directly to binary. If you could show me how to do that, it would solve my biggest problem. Here are some examples of what I would call complex types: ```auto public type Address = { id: Nat32; street1: Text; street2: Text; city: Text; province: Text; postalCode: Text; country: Text; }; public type Person = { id: Nat32; firstName: Text; lastName: Text; emails: List; addresses: List; }; ``` ---
**Author:** ![Motokoder](https://avatars.discourse-cdn.com/v4/letter/m/e47c2d/32.png) [@Motokoder](https://forum.dfinity.org/u/Motokoder)\ **Post date:** [December 31, 2021, 4:32am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/25 "2021-12-31T04:32:49Z")
@timo @jzxchiang It looks like the [inter-canister message size limit](https://github.com/dfinity/ic/blob/391fd19f2154471f01068aaa771084eac010a099/rs/types/types/src/messages.rs#L39) is 2 MB. ---
**Author:** ![timo](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/timo/32/5543_2.png) [@timo](https://forum.dfinity.org/u/timo)\ **Post date:** [December 31, 2021, 7:44am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/26 "2021-12-31T07:44:50Z")
> [@jzxchiang](#): > > do you mean to use the `loadBlob` [function](https://smartcontracts.org/docs/base-libraries/ExperimentalStableMemory.html#loadBlob) Yes, I meant that one. ---
**Author:** ![timo](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/timo/32/5543_2.png) [@timo](https://forum.dfinity.org/u/timo)\ **Post date:** [January 1, 2022, 8:59pm UTC](https://forum.dfinity.org/t/motoko-serialization/9954/27 "2022-01-01T20:59:40Z")
> [@Motokoder](#): > > Here are some examples of what I would call complex types: > > ```auto > public type Address = { > id: Nat32; > street1: Text; > street2: Text; > city: Text; > province: Text; > postalCode: Text; > country: Text; > }; > > public type Person = { > id: Nat32; > firstName: Text; > lastName: Text; > emails: List; > addresses: List; > }; > > ``` For the text field you could serialize to binary by writing the length as one word followed by the text as Blob. For example define a function ```auto import S "mo:base/ExperimentalStableMemory"; import Nat32 "mo:base/Nat32"; import Text "mo:base/Text"; func storeText(offset : Nat32, value : Text) : Nat32 { let b : Blob = Text.encodeUtf8(value); let len : Nat32 = Nat32.fromIntWrap(b.size()); S.storeNat32(offset, len); S.storeBlob(offset+4, b); 4 + len }; ``` and then call it like this: ```auto offset += storeText(offset, street1); offset += storeText(offset, street2); offset += storeText(offset, city); ... ``` ---
**Author:** ![timo](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/timo/32/5543_2.png) [@timo](https://forum.dfinity.org/u/timo)\ **Post date:** [January 1, 2022, 9:01pm UTC](https://forum.dfinity.org/t/motoko-serialization/9954/28 "2022-01-01T21:01:13Z")
This example runs in the playground at the moment. However, I think the interface for ExperimentalStableMemory has recently changed to 64 bit address space so you may have to change all Nat32 to Nat64, depending on your environment. ---
**Author:** ![Motokoder](https://avatars.discourse-cdn.com/v4/letter/m/e47c2d/32.png) [@Motokoder](https://forum.dfinity.org/u/Motokoder)\ **Post date:** [January 2, 2022, 12:35am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/29 "2022-01-02T00:35:41Z")
Thank you @timo. That was very nice of you to provide a code sample. It really cleared up my confusion. Happy New Year! ---
**Author:** ![mbeekman](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/mbeekman/32/2675_2.png) [@mbeekman](https://forum.dfinity.org/u/mbeekman)\ **Post date:** [July 3, 2022, 10:50am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/30 "2022-07-03T10:50:32Z")
This is a great thread and directly related to a problem I want to solve … As @jzxchiang asked and @timo replied with: > [@timo](#): > > To back up the data in the stable memory I would just take the entire memory pages and ship them off as Blob to another canister. That way the type transmitted is always Blob and the backup canister can be completely agnostic to the content and the types used in the content (as you wanted). You should be able to send at least 2 MB in a single inter-canister message. With the stable memory being organized in memory pages of 64kB you can then ship 32 pages at once. I am trying to do just that … not sure if there are examples of that somewhere ? and on the same subject, lets assume I can figure out how to create the Blob I want and can break it into 2MB chunks of pages… Can you turn an array of complex objects into a Blob relatively easily? … if I were to take the example from @Motokoder, i.e. Address, and had an array of addresses i.e.: ```auto var theAddresses : [Address] = [] ; ``` and then I wanted to turn “theAddresses” into a Blob so I can chunk it up and send to another canister … I feel like I am 80% to the finish line, but still a little confused … any assistance would be great. Thanks in advance … and I know this thread is a little old, but like I said, right on target with what I am trying to do … ---
**Author:** ![icme](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/icme/32/4327_2.png) [@icme](https://forum.dfinity.org/u/icme)\ **Post date:** [July 3, 2022, 3:30pm UTC](https://forum.dfinity.org/t/motoko-serialization/9954/31 "2022-07-03T15:30:20Z")
Check out the `to_candid` and `from_candid` methods introduced in the links provided in [this post](https://forum.dfinity.org/t/icdevs-org-bounty-18-cbor-and-candid-motoko-parser-3-000/11398/5) ---
**Author:** ![mbeekman](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/mbeekman/32/2675_2.png) [@mbeekman](https://forum.dfinity.org/u/mbeekman)\ **Post date:** [July 5, 2022, 7:42am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/32 "2022-07-05T07:42:48Z")
Holy guacamole … seems to be exactly what I am looking for … hard to find, but for the next person along the same path … here is a direct to the doc on it 🙂 > **[Language quick reference | Internet Computer Home](https://internetcomputer.org/docs/current/developer-docs/build/languages/motoko/language-manual/)** > > This section serves as a technical reference for the previous chapters and has specific technical information for readers with specific interests. For example, this section provides technical details of interest to the following audiences: (fyi - the forum strips the #candid-serialization … search for “Candid Serialization” in this page … ) Muchas gracias @icme (a.k.a. Canscale) … and again thanks to @jzxchiang @timo 🙃 going to try it right now … ---
**Author:** ![mbeekman](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/mbeekman/32/2675_2.png) [@mbeekman](https://forum.dfinity.org/u/mbeekman)\ **Post date:** [July 5, 2022, 8:03am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/33 "2022-07-05T08:03:14Z")
Also found this helpful as a reference … > [@Candid Serialisation: to\_candid and from\_candid](https://forum.dfinity.org/t/candid-serialisation-to-candid-and-from-candid/13543): > > In the light of the new candid serialisation I thought I’d share a simple example to serialise/deserialise a type. The debug will print: [Canister rrkah-fqaaa-aaaaa-aaaaq-cai] ?{num = 1; text = "test"} Hope this help others. thanks @Gabriel🙂 ---
**Author:** ![mbeekman](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/mbeekman/32/2675_2.png) [@mbeekman](https://forum.dfinity.org/u/mbeekman)\ **Post date:** [July 5, 2022, 8:12am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/34 "2022-07-05T08:12:17Z")
and thanks to @matthewhammer and @claudio for building it in the first place (fyi … need to upgrade to 0.10.1 to get the Motoko version with the methods … otherwise will get “unbound variable to\_candid” ) … rocking on … ---
**Author:** ![mbeekman](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/mbeekman/32/2675_2.png) [@mbeekman](https://forum.dfinity.org/u/mbeekman)\ **Post date:** [July 10, 2022, 11:53am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/35 "2022-07-10T11:53:17Z")
As a closer to this subject … the from\_candid method returns an optional type or () (?Type) … this was a new thing for me, as it will not compile if you take that value and try to use it … so we need a switch to check whether it worked … even though it will assert if the “from\_candid” fails … here is a quick code example that may prove useful to the next Explorer walking this path: ```auto var tempBlobReturned : ?ArchiveType = from_candid(tempBlob); switch tempBlobReturned { case (?val) { Debug.print("RESTORE SUCCESS - have a value" ) ; tempArchiveRestored:=val ; }; case null { Debug.print("RESTORE FAILED - have no value " ) ; }; };// end switch ``` Happy to respond if anything does not make sense here … extrapolated this solution from the following post: > [@Сannot producer expected type](https://forum.dfinity.org/t/annot-producer-expected-type/5544/2): > > You need to switch and pattern-match on the optional value: switch result { case (?val) { .../\* val : Text here\*/... }; case null { ... }; } If you have multiple of such tests, take a look at the do? construct, which comes in handy and e.g. allows collapsing multiple switches into one: let result = do? { List.last(list1)! + List.last(list2)! + 1 }; switch result { case (?n) { ... handle result ... } case null { ... handle error ... } } The ! operator inside a do? block converts ?T t… needless to say, but very obtuse … thanks @rossberg … please rock on … ---
**Author:** ![ImSteffi](https://sea1.discourse-cdn.com/flex023/user_avatar/forum.dfinity.org/imsteffi/32/19792_2.png) [@ImSteffi](https://forum.dfinity.org/u/ImSteffi)\ **Post date:** [April 27, 2025, 2:11am UTC](https://forum.dfinity.org/t/motoko-serialization/9954/36 "2025-04-27T02:11:54Z")
i wonder if this guy is still with us xd [Previous page](https://forum.dfinity.org/t/motoko-serialization/9954.md?page=1)