How to maintain nested hashmaps as stable

Let’s say I create hashmap where every key is assigned to another hashmap, and I populate this with some values:

var x1 = H.HashMap<Text, Nat>(2, Text.equal, Text.hash);
var x2 = H.HashMap<Text, Nat>(2, Text.equal, Text.hash);
var map = H.HashMap<Text, H.HashMap<Text, Nat>>(10, Text.equal, Text.hash);
        
x1.put("a1", 50);
x2.put("a1", 100);
map.put("2021-09-22", x1);
map.put("2021-09-23", x2);

How do I go about maintaining the state of this variable? I understand that I’m not able to declare the map variable as stable, and I’ve taken a look at pre-upgrade and post-upgrade functions, but I’m having trouble converting this hashmap into the correct kind of stable list.

So far I’ve tried iterating through that map variable in an attempt to write it to a list of type [ (Text, [ (Text, Nat) ] ) ], but I haven’t had any success so far. I’m not used to declaring list sizes in advance, so this is giving me some trouble. Is there an easy way to do this, or is there a different way I should be thinking about this?

You can create a stable array with the same types as your hashmaps and indeed use pre-post upgrade.

For example this playground

private var canisters: HashMap.HashMap<Principal, CanisterId> = HashMap.HashMap<Principal, CanisterId>(10, isPrincipalEqual, Principal.hash);

private stable var upgradeCanisters : [(Principal, CanisterId)] = [];

system func preupgrade() {
        upgradeCanisters := Iter.toArray(canisters.entries());
    };

    system func postupgrade() {
        canisters := HashMap.fromIter<Principal, CanisterId>(upgradeCanisters.vals(), 10, isPrincipalEqual, Principal.hash);
        upgradeCanisters := [];
    };
1 Like

On preupgrade you call Iter.toArray to transform / populate the HashMap to the stable Array.

On postupgrade you call the HashMap.fromIter to transform / populate the stable Array to HashMap.

Hi @peterparker, I’ve tried something similar, but it doesn’t work for the code sample I’ve posted above because the hashmap I’m converting to an array contains another hashmap. Going off the initial example I posted:

var map = H.HashMap<Text, H.HashMap<Text, Nat>>(10, Text.equal, Text.hash);
stable var upgradeMap : [(Text, H.HashMap<Text, Nat>)] = [];
// ...
system func preupgrade() {
    upgradeMap := Iter.toArray(map.entries());
};

Would generate an error:

main.mo:21.16-21.28: type error [M0131], variable upgradeDates is declared stable but has non-stable type
  [(Text, HashMap<Text, Nat>)]

I have also tried defining upgradeMap as follows:

stable var upgradeMap : [(Text, [(Text, Nat)])] = [];

But this still produces an error:

main.mo:38.23-38.50: type error [M0098], cannot implicitly instantiate function of type
  <A>(Iter/1<A>) -> [A]
to argument of type
  Iter/1<(Text, HashMap<Text, Nat>)>
to produce result of type
  [(Text, [(Text, Nat)])]
because implicit instantiation of type parameter A is over-constrained with
  (Text, HashMap<Text, Nat>)  <:  A  <:  (Text, [(Text, Nat)])
where
  (Text, HashMap<Text, Nat>)  </:  (Text, [(Text, Nat)])
so that no valid instantiation exists

I’m pretty sure I understand why the errors are occurring, I’m just not familiar with the proper syntax to apply Iter.toArray() in scenario like this, where I have a hashmap inside a hashmap

Actually, I also have got such an example in my repo :wink:. Not sure that the best solution but, it works out for me.

From my actor class functions pre/postupgrade I call another class that contains following functions:

public func preupgrade(): HashMap.HashMap<UserId, [(DeckId, OwnerDeckBucket)]> {
            let entries : HashMap.HashMap<UserId, [(DeckId, OwnerDeckBucket)]> = HashMap.HashMap<UserId, [(DeckId, OwnerDeckBucket)]>(10, Utils.isPrincipalEqual, Principal.hash);

            for ((key: UserId, value: HashMap.HashMap<DeckId, OwnerDeckBucket>) in decks.entries()) {
                let ownerDecks : [(DeckId, OwnerDeckBucket)] = Iter.toArray<(DeckId, OwnerDeckBucket)>(value.entries());
                entries.put(key, ownerDecks);
            };

            return entries;
        };

        public func postupgrade(entries: [(UserId, [(DeckId, OwnerDeckBucket)])]) {
            for ((key: UserId, value: [(DeckId, OwnerDeckBucket)]) in entries.vals()) {
                let ownerDecks: HashMap.HashMap<DeckId, OwnerDeckBucket> = HashMap.fromIter<DeckId, OwnerDeckBucket>(Iter.fromArray<(DeckId, OwnerDeckBucket)>(value), 10, Text.equal, Text.hash);

                decks.put(key, ownerDecks);
            };
        };

My idea is to convert first the HashMap<Something, Hashmap<Something, Something>> to HashMap<Something, [(Something, Something)])> before converting it to the stable array on preupgrade and the reverse operation on postupgrade.

Downside of the approach is that the upgrade might become slow over time as I iterate over the hashmap on upgrades to convert each sub-hashmaps to arrays but, like I said, does the job.

Let me know if that would work for you?

1 Like

This is exactly what I was looking for, thank you! I was trying to do the same thing, but I’m still getting used to the Motoko syntax. This was very helpful

Me too!

Happy to hear it helped and let me know if it works out.

It did wind up working out using similar syntax as your example. I can see this being slow if the hashmap gets too large, but for now, I don’t expect to upgrading too frequently

1 Like

Cool thanks for the feedback

I would recommend first converting the outer hashmap to an array and then map over that. That way, you only allocate one intermediate array instead of an intermediate hashmap, which is more expensive.

You can also do it in one pass, by creating and initialising the target array manually instead of using toArray.

1 Like

Sounds super interesting! Don’t you happen to have some sample code to display?

Something along the lines of (untested):

var map = H.HashMap<Text, H.HashMap<Text, Nat>>(10, Text.equal, Text.hash);
stable var upgradeMap : [var (Text, [(Text, Nat)])] = [var];

system func preupgrade() {
    upgradeMap := Array.init(map.size(), ("", []));
    var i = 0;
    for ((x, y) in map.entries()) {
      upgradeMap[i] := (x, Iter.toArray(y.entries()));
      i += 1;
    };
};
2 Likes

Nice! Thanks for the snippet, gotcha.