Ic-stable-structures - page size

Developers
1

Hi

I’m developing an archive canister for storing records.
I use 2 x BTreeMap from ic-stable-structures.

#[cfg(feature = "inttest")]
const MAX_INFO_SIZE: u32 = 28500;

#[cfg(not(feature = "inttest"))]
const MAX_INFO_SIZE: u32 = 1000;

impl Storable for Info {
    fn to_bytes(&self) -> Cow<[u8]> {
        Cow::Owned(Encode!(self).unwrap())
    }
    fn from_bytes(bytes: Cow<[u8]>) -> Self {
        Decode!(&bytes, Self).unwrap()
    }
    const BOUND: Bound = Bound::Bounded {
        max_size: MAX_INFO_SIZE,
        is_fixed_size: false,
    };
}

impl Storable for ID {
    fn to_bytes(&self) -> Cow<[u8]> {
        Cow::Owned(Encode!(self).unwrap())
    }
    fn from_bytes(bytes: Cow<[u8]>) -> Self {
        Decode!(&bytes, Self).unwrap()
    }
    const BOUND: Bound = Bound::Bounded {
        max_size: 100,
        is_fixed_size: false,
    };
}

#[derive(Serialize, Deserialize)]
pub struct Archive {
    #[serde(skip, default = "init_map_records")]
    recorsd: StableBTreeMap<ID, Info, VM>,
    #[serde(skip, default = "get_user_swap_id_map")]
    user_swap_id_map: StableBTreeMap<Principal, Vec<u32>, VM>,
}

As you can see, nothing special right now. The only thing to note is that in integration testing we set the flag “inttest” to force the pagesize to be artificially high so that we can test what happens with more archive canisters deployed.

When i calculate the byte size of ID and Info, i get a total size of 392. but because we set it to a max size of 1000 i believe this is the size that is used by BTreeMap

I know that when the archive canister is initailized it will take up 8mb for both BTreeMaps and as such the archive is spun up with around 16mb of memory.

to test in a staging ( live ic canister ) we say that if any archive canister has more than 22mb of memory then a new archive canister gets created. this should leave us with enough room for the first 8mb of Info records, at around 8k records. but we’re seeing a new archive created at only the 27th Info record. this would imply that either new pages being created when its not needed. or we’re making a mistake somewhere.

I notice that after upgrading the canister and then using the canister, it tends to trigger a new archive canister to be created. its almost like upgrading creates new pages

ic-stable-structures = “0.6.4”

Any ideas?

2

I see 100 set in the code snippet you shared. Which is true? You use 1000 or 100?

This indeed sounds not expected. I’m not sure I have a good guess off the top of my head.

Also sounds a bit weird if it’s indeed what’s happening.

I think we would need a minimal reproduction so we can further look into it and debug what’s happening. Would you be able to create a small repository with the minimal example code and steps to reproduce the issues you noticed and then share here?