Increased Canister Smart Contract Memory

Oh, and more more issue with the proposed spec: the code for stable_grow must return -1 if the new size would be above 2^32 bytes. The current code would allow a 32 bit API using canister to grow the canister above 4GB (and then start trapping in every call).

1 Like

In all seriousness, thank you for taking the time to read in earnest, Joachim

4 Likes

@ulan @akhilesh.singhania

2 Likes

Thanks @akhilesh.singhania for your detailed technical explanation of this change. Thanks @diegop for your follow-through and involving different teams. I do think that proposal adds significant value to the development of IC; with the increased canister storage.

I agree, in part, with @lastmjs on the voting issue with Dfinity Foundation. i.e. I believe that Dfinity Foundation should abstain from voting (even if the proposal fails); even if there is no mechanism to ban Dfinity Foundation from voting.

Democracy, even liquid democracy, assumes (nay requires) that all voting participants are equally informed about all issues surrounding the topic that they are interested in voting on. In software development, it is NOT sufficient to just to show a diff to the voters who do not have access to the source code versus others (in Dfinity Foundation and others bound by NDAs) who do. In asking for such a vote, we are inadvertently creating different classes of voters; one class that is not completely informed (unless the source code is open sourced) and one class that is informed through access to source code. By not providing access to Unit and Integration Test cases, we are once again creating a divide. We are saying to the folks that do not have access, please trust the ones that do have access in their testing.

I disagree, in part, with @lastmjs in context of how bigger players have bigger say in the voting. This is the underlying structure that I believe that I agreed to when I decided to participate in the IC eco-system. I understood my neuron may not have the same voting power as someone else’s neuron. I am ok with it.

The notion of abstinence in voting should certainly not alien; especially since the source code is expected to be open-sourced at some time.

2 Likes

Regarding alternative 1: isn’t this fairly easy to route around via an upgrade? Upgrade a 32 bit canister to include export routines to move data to a trusted backup canister, upgrade to a 64 bit canister with an empty memory that has rehydrate functions to pull data back. Then upgrade and remove the hydration functions?

If it is cleaner it may be worth making existing 32 bit canister jump through this while we only have a relatively small number.

I’m all for the upgrade, but think we should run out all the options and beat them up. We don’t want to generate a bunch of needless complexity.

These are all very reasonable points, and very compelling ones!

I am a big believer in “laying out the Legos on the table and seeing what we can build together.”

With that in mind, there is an additional lego I want to lay out on the the well-reasoned table you’ve set:

I think there is a silent group of people who TRUST and want the foundation to make technical choices (they make this choice by following the neurons). For the foundation to NOT vote would be to rob this group of some weight or say.

Are there people who follow the foundation out of sheer default? Very likely. But I also think there are many earnest folks who express their will through following. I have been thinking a lot about this group and how to make sure they are also heard.

I don’t have all the answers, but I wanted to lay out all the Legos on the table…

4 Likes

Thanks, @diegop for your Lego Block. My concern, per se, is not about liquid democracy. In fact all four of my neurons follow the foundation and therefore I trust the foundation to make the correct decision on my behalf. As a matter of fact, I don’t have the time to go through the code that I am asking to be shown. Again I signed up for liquid democracy when I decided to invest in the IC eco-system. I am ok with it.

However in our fledgling liquid democracy experiment, it is important to call out issues that need to be called out for others as well. A potential future resolution could be to NOT bring a proposal prior to all the source code being viewable ; if approval is required on the code itself.

I anticipate that this issue, while fairly innocuous now, may become quite contentious in the Threshold EDSCA Signature case. This is because Dfinity Foundation might WANT to hang on to the source code and the papers in private; in the belief that this gives IC a leg up visa-ve competitors. But then how does one get the community to vote on it as an informed citizenry; especially the community will need to trust this code for IC-BTC integration?

3 Likes

This would be awesome!

Well, for the way Motoko uses stable memory now, e.g. as the backend for the fully automatically managed stable variables, using the new API is pretty simple, but also not very useful, as you still need the stable data in main memory between upgrades, so still bound by 4GB.

Changing the implementation of stable variables to, say, load them on demand would allow Motoko to hold more data and have many other benefits (e.g. no out-of-cycles worries when upgrading), but would be a bigger undertaking.

A middle ground is giving developers mostly raw access to stable memory. @claudio is working on that (design doc). 32 vs. 64 won’t make a big difference here, but it’s still WIP.

3 Likes

Good catch! That was the actual intention.

Another good catch! Thanks a lot. I’ll fix the spec diff today/tomorrow.

1 Like

Boom! Open design process :slight_smile:

These are all great points, and ones I struggle with internally (as I weigh cons/pros and trade-offs) to be honest, I don’t feel I have clarity of thought (yet) to add anything, and I am a big believer in “whereof one cannot speak, one must remain silent”, I have no great need to be clever, write for sake of writing, or win arguments, so you should know that I am certainly listening and absorbing. Let me brew on this a bit, but definitely know that this process is meant to be iterative and interactive. We will adapt.

Which source code are you talking bout that’s not open ?

The asciidoc sources of the Interface Specification

1 Like

FWIW I have chosen to follow the DFINITY foundation’s neuron, and I do not want them to abstain from voting. Isn’t that the whole point of vote delegation? I trust the foundation’s decisions on these early, foundational proposals. If at some point I disagree with the foundation’s decisions, I will manually vote, or delegate my votes to someone else.

4 Likes

I AM one of these people. I trust in Dfinity the same way I trust in my doctor. I might not understand every little decision that they take for my well being but I trust in their expertise to make the best educated guesses even in cases of uncertainty.

1 Like

I updated the proposed spec changes to address @nomeata’s comments. Here is the diff to what @akhilesh.singhania posted initially.

2 Likes

LGTM, as far as I can tell at 11pm.

3 Likes
6 Likes