Hi @skilesare. While I am not completely sure about what happened in the particular example you cited (and we are looking into this), I do want to respond more generally about API stability.
I first want to say that API stability is very important. The NNS team recognizes the need to keep APIs stable.
But I want to discuss why we think it’s important, and give some more context for our decision-making.
Platform Adoption as Success Metric
I would argue that API stability is a value resulting from a very practical consideration around platform adoption.
If your frontend code is the only user of your API, and you have no desire to support other users, you could change it at will in whatever way makes the rest of the application easier to develop.
However, when an API is public, and starts to be used by many parties, API stability becomes important because developers will not develop against an API if it keeps changing. It is disruptive to the ecosystem.
As IC community members, we want the ecosystem to thrive, and we want to avoid any disruptive changes that require a response by the community as much as possible. You pointed out some circumstances where that might be impossible (legal or security issues that somehow require breaking changes). But we want it to be the exception, if not something that simply doesn’t happen.
However, just as importantly, developers won’t develop on a platform if it’s stagnating and no new features are being released. The IC is providing support for increasingly sophisticated use cases, and as a cutting edge technology which is gaining more and more adoption, this also must be considered critical. If it does not, it will not continue gaining traction, as it has.
So what do you do with competing values?
Dealing with Competing Values
In the case mentioned, we realized at the time that adding that variant could cause problems with Candid decoders, as there was an issue that prevented the opt variant from decoding as null as it should. (I thought that issue was fixed in more recent versions (which would explain why one of your motoko playgrounds decoded null correctly)).
We had to decide between some unpleasant options: delaying the SNS One Proposal feature to do a substantial amount of architectural work, or making a change that could have an impact on API clients.
Given the importance of continuing to develop the SNS for the overall ecosystem’s health (so far have 9+ projects that have successfully launched an SNS), it seemed that delaying would cause negative repercussions.
To account for the impact to users, we announced in April that there would be a breaking API change caused by this work, and what downstream clients needed to do in order to avoid being impacted.
Due to the nature of that change, no clients would actually be affected until a proposal of the new kind CreateServiceNervousSystem was created and the client attempted to decode it.
In effect, this meant that for any developer who noticed the change, they had about 4 months to make changes. This was rolled out in this way to try to have as little impact as possible on applications on the IC.
What we could have done better
Where there seems to have been a disconnect is that some projects didn’t get the information in time, and ICDevs seems to have been affected in this way.
We had decided to use the forum, so that it would be publicly visible, but maybe that was not enough.
Moving forward, we will have a single thread that announces all of the NNS Updates with breaking changes, so that developers can subscribe to a single post and get notifications.
We are open to other suggestions though. What, in your opinion, would help to make sure that upcoming changes are easier to discover and be notified about?
Balancing Long-term API Stability and Changes
Our team recognizes that what happened with 1-proposal is not ideal. This has happened in a few other cases where new proposals had to be added to the NNS. I hope this is now resolved at the Candid decoding level so that variants can be safely added.
However, we have been questioning (and long before this post brought it up) the original architectural decisions made in NNS canisters that make the APIs hard to change independently of the application logic of the canisters. As opportunities allow, we will be addressing those shortcomings to further reduce disruptions, ideally to zero, of any changes except those dictated by security or legal concerns (which also generally can be done without breaking clients).
We agree with your observations about how endpoints should evolve, in a backwards compatible way, and about versioning endpoints to add new functionality. The NNS team has been thinking and discussing what we would like to do in the NNS along the same lines this year already.
I think, as my personal opinion, that some APIs will eventually have to be deprecated. There is no way to know how different the world will be in 10 years, and while we could, in theory, never change an API, that has to be balanced with letting the platform stagnate and not meet the needs of developers 10 years hence.
That’s a discussion for the community as it moves forward and decides what the future of the IC looks like. It’s a platform built to allow evolution and community ownership.
Because the NNS is a DAO meant to allow the IC to change at the pace of the internet, to be able to support uses that today are unimagined, you might expect that there will eventually be changes that prevent it from being interoperable with very old applications that never update.