Introduction
If you came for the ICP giveaway, youāll need to read to the end (and itāll likely save you time if you read and absorb this post first 
Iāve tried my best to shorten it). If you care about the sanctity of IC governance and desire effective decentralisation then I think you should read to the end 
Context
This post stems from prior discussion on a recent IC-OS proposal forum topic (along with prior discussion on Slack and OC) highlighting proposal summary inaccuracies and debating the unnecessary difficulty involved in validating proposal summaries.
This post also comes at a time where CodeGov may stop conducting technical reviews on IC-OS proposals, given that the funding and decentralisation landscape is changing. As a member of CodeGov, I see this post as an opportunity to consolidate and summarise some concerns that Iāve been raising over the last few months while conducting IC-OS proposal reviews.
See here if youāre unclear about whatās meant by IC-OS (the operating systems that run the IC - i.e. IMPORTANT stuff). Anybody whoās spent any length of time digging into the details presented by IC-OS proposal summaries (and then digging deeper) will know that theyāre not always what they seem.
Overview
In this post I aim to:
- Introduce IC-OS proposal summary inaccuracies
- Provide my understanding of why these occur relatively frequently
- Explain why I thinkā¦
- ā¦ current approaches to addressing this problem are insufficient (tackling symptoms rather the cause)
- ā¦ this problem is far from optimal for effective governance decentralisation (which the IC is increasingly expected to demonstrate)
- ā¦ this requires a rethink of how the IC repo is managed (tackling the cause rather than the symptoms, saving everyone time and effort in the long run)
Iāll finish off by proposing some simple next steps as well as requesting discussion and feedback from DFINITY and the rest of the community about the feasibility and utility of what Iām suggesting.
IC-OS Proposal Inaccuracies
As it stands, ICP stakers are invited to vote on IC-OS proposals that the proposer (DFINITY) hasnāt managed to maintain sufficient oversight of to accurately summarise (in a non-misleading manner). Here are a couple of recent examples where relevant commits have been left out of the summary, or irrelevant commits have been included - 130984, 130818. Another type of inaccuracy references an incorrect commit as the start of the change log - here are a couple of examples, 131055, 130749.
Straw that broke the camelās back
Individually small discrepancies may seem harmless, but misleading proposal summaries collectively have the potential to be dangerous (itās not something we should be normalising and accepting). Iāve recently started rejecting IC-OS proposals that have misleading summaries (
thank you @wpb for being receptive to this).
Note that if the community rejected IC-OS proposals every time the proposal summary is misleading (e.g. missing out relevant changes and/or including irrelevant changes) I would not be surprised if the majority would not be executed. Nevertheless, Iām confident that this is where effective governance decentralisation begins.
Proposal summaries currently risk doing more harm than good by eroding trust in the integrity of IC proposal history (history which is expected to have been comprehensively validated by at least some members of the community before execution).
Detecting Missing Commits
Regarding the problem of missing commits, I generally detect these by looking for files that have been modified by commits referenced in the proposal summary, and then looking for commits that werenāt referenced by the proposal summary but that also modified these files. This throws up a load of false positives (mostly cargo and bazel related files, or genuinely unrelated files due to fat commits), so I then sift through these manually in an attempt to pick out genuine IC-OS modifications.
Short of paying very close attention to the hundreds of commits that take place between GuestOS election proposals, and the thousands of commits that take place between HostOS election proposals (with a limited time budget) this seems like a sensible way of short-listing missing commit candidates (which should have been referenced by the proposal summary in question). Iāll come back to this point in the next section about so many unrelated commits to sift through (I believe itās unnecessary, and costly to the objective of governance decentralisation).
Other Approaches
The approach Iāve described above is obviously far from perfect. Another approach would be to utilise the CODEOWNERS lookup file, which maps out which teams are responsible for which files in the IC repo. This is one of the techniques used by the tool that actually generates IC-OS proposal summaries. However this is also imperfect, and assumes that the CODEOWNERS file is accurate and kept up-to-date (which I donāt believe is often the case).
If proposal summaries cannot be depended on (in the sense of being easily validated to be true and accurate), their utility is brought into question. I donāt believe thereās a good reason for accurate and easily verifiable IC-OS proposal summaries to be difficult (to produce, nor to verify), as Iāll explain.
Why Do Inaccuracies Occur So Frequently?
Let me be clear about what Iām saying. DFINTY is doing an unparalleled job of building something incredible and world changing. My hat goes off to them. Iām extremely grateful for the work that theyāre continuing to do expertly. The root of the problem doesnāt lie with the DRE team nor the proposal summary tooling - I think theyāre forced to fight a loosing battle against how the IC repo is being maintainedā¦
As I see it, the problem stems from a choice of how the IC repo is maintained to reduce certain burdens on the development teams (requiring voters/reviewers in the community to work many times harder to work around that choice). Itās a trade-off, but the community is increasingly forced to pay for it. In my opinion itās a choice that only pushes complexity around, kicking the can down the road where it becomes harder to account for.
The Choice of Trunk-based Development
Iām speaking about the choice to practice whatās called ātrunk-based developmentā. Youāll need to be familiar with Git to grasp this. In a nutshell, trunk-based development assumes untenable merge conflicts are practically unavoidable when performing development work on separate git branches. It seeks to avoid these merge conflicts by ensuring that branches are merged into a single trunk (master branch) as frequently as possible (so that any merge conflicts either do no exist, or are small and easier to resolve). IC branches often contain only 1 commit.
Note that the IC repo contains many projects (deployment targets that are handled by different proposals and/or different types of proposals), including system canisters and other system components such as GuestOS and HostOS. Trunk-based development mixes commits for all of these systems together into a big jumble on the master branch (or trunk) before theyāve even been deployed/approved by the community.
This is a jumble thatās evidently hard to untangle when composing/reviewing releases. Trunk-based development is clearly a bad fit for a Web3 foundation (probably why Google doesnāt see an issue with practising trunk-based development - theyāre a different enterprise with different trade-offs to juggle).
Whatās the point of a branch if it contains only 1 commit?
Branches that contain only 1 commit are largely pointless (many popular git clients provide the option to automatically ākeep a single branch line if possibleā to avoid such pointless branches cluttering the git graph).
Git is a powerful and flexible fully fledged source control system supporting numerous branches for features, environments and deployment targets (with no end of tooling for working with and visualising the resulting git graph). In this context, trunk-based git development is a bit like using a broom to paint a masterpiece, as you donāt want to have to keep cleaning your job-specific, assorted paint brushes (even though theyāre there to be used). Youāll get the job done and cover more ground, but there will be mistakes in the presentation of your work, and the audience will be left wondering what theyāre looking at (repository-wise).
Rethinking the IC Repo 
Itās DFINITYās contention that trunk-based development is a practical necessity that pays for itself by avoiding merge conflict headaches. Iām not convinced by this. I am convinced that thereās huge cost thatās being paid by the community and the mission for governance decentralisation.
The fact that DFINITY struggle to compose their own IC-OS proposal summaries accurately should be seen a red flag, indicating that something fundamental needs to change. How can the community be expected to step up and take responsibility for proposals when even the proposer often fails to do so accurately. This goes deeper than just tweaking tooling and dependency graphs (a current approach for addressing this - e.g. see here).
We should be minimising the number of tools that the community needs to (or is inclined to) place trust in. Aside from build/hash verification, Git is the only source of truth that should be needed, and I think it can and should be made easy for the community to validate the integrity of a proposal summary based on the content of a release branch (a branch that does not contain any irrelevant commits, and therefore is listed in itās entirety in the proposal summary).
Proposed Next Steps
I strongly believe that a long-term goal should be making IC-OS proposal summaries something that is validated in a similar fashion to build verification. This sort of thing can and should be simple, but the IC repo needs to be managed differently for it to be feasible.
Here are some quick wins and low hanging fruit:
- Committers begin listing the deployment targets that their commit modifies in the commit message, using a standardised format on a single line (e.g. āTarget: GuestOS, HostOSā or āTarget: GuestOS, RegistryCanisterā)
- I also think it would be useful to explicitly distinguish between a āDirect Targetā (e.g. directly modifying GuestOS code) and āIndirect Targetsā (e.g. modifying/upgrading a library that indirectly impacts the behaviour of the GuestOS, as a dependant system)
- In the case of āDirect Targetā commits, committers should aim to limit commits to a single deployment target (split changes between separate commits if needed/possible - e.g. donāt directly modify GuestOS and HostOS in the same commit)
- In the event that this isnāt possible, provide an explanation for why deployment targets are so coupled that separate commits would not be feasible/sensible
- Committers should aim to not mix āDirect Targetsā with āIndirect Targetsā in the same commit (e.g. do not modify common library code thatās depended on by GuestOS in the same commit that directly modifies GuestOS code). This might seem odd to anyone whoās bread and butter is trunk-based development, but itās what makes commits composable into different branches (it doesnāt necessarily matter if one commit doesnāt represent a completely working state if itās not merged into master yet).
- @DRE-Team can then utilise these Target labels to quickly and reliably construct IC-OS proposals (and/or use existing proposal summary tooling to detect most cases where the committer is not abiding by these guidelines, so that they can be reminded for next time)
The above points serve the following purposes:
- Quickly get to a point where IC-OS proposals more reliably contain all of the relevant commits, and none of the irrelevant ones (though thereās still the problem of simple and reliable verification, which is something to tackle later - see next point)
- Acquire clearly interpretable quantitative and qualitative data about how frequently cross-cutting concerns and common dependencies result in a commit that directly modifies multiple deployment targets, and in those cases, information about why the commit cannot sensibly be decoupled into separate commits for the separate deployment targets (this is provided by all of the commit messages).
- Note that this same constraint wouldnāt be expected to be upheld by commits for āIndirect Targetsā (common library/utility code), because itās purpose is to be shared between projects.
Once/if the above could be ticked off, as a community itāll be a lot easier to engage in further discussion about how matters could be improved, and git workflows and processes that are more conducive to an inclusive and active NNS DAO and Web3 community (where proposal summary verification is made simple due to deployment-target-specific release branches).
I think the IC is well positioned to explore and set a gold standard for how source control for Web3 projects are managed in the future. We just need to want this - do we?
Wrapping Up
In the meantime, Itās my contention that the community should be aiming to reject IC-OS proposal summaries that are incorrect or miss out commits that modify the relevant IC-OS (I will be aiming to do this from now on, as mentioned). This is my public neuron if youād like to follow it for this purpose (IC OS Version Election) - 16459595263909468577. I plan to make this a known neuron at some point.
Please voice your opinion in comments, ask questions, express if you agree or disagree with points raise in this post, and what it would take to move forward with these ideas. @Sat, @Luka, @EnzoPlayer0ne and @basvandijk, Iād be particularly interested to know what your thoughts are if you have the time
Summary
In summary, the current approach to managing the IC repo creates significant challenges for accurate proposal summaries, and proposal summaries that can be easily validated, which in turn hampers effective governance decentralization. By adopting the proposed steps, we can inch towards making the process more reliable, and easier for the community to engage with.
I urge the community to consider showing their support for these simple changes to commit messages, in aid of exploring a more robust and trustworthy governance workflow. Please share your thoughts, questions, and feedback to help us move forward together.
As a side note (expand for details)
As a side note, some other things that I think would help the community get more of a grasp on releases:
- Referencing the merge commits in the proposal summary, as these commit messages are always far more informative (this would help community members who would benefit from easy access, rather than having to navigate the git graph to get this information). Requested here. Big thanks to @Sat for making this happen!
- Public pull requests (requested previously and announced on Slack today). Big thanks to @basvandijk and the IDX team for pulling this off!
- Awaiting further feedback about the feasibility of making Github issues public (at least where there are no security concerns)
- Commits messages that always provide a why, and not just a what (the latter can always be gathered from the changes being made, the former canāt). Given that these commit messages are the first window into what the proposal is doing (and the community is being asked to vote on it), the more context and information these commit messages can provide the better. Others in the community already make it their mission to validate the accuracy and integrity of these commit messages (but theyāre sometimes lacking in context - other times theyāre great).
- Reference the original commit in the commit message when cherry picking (most git clients can be configured to do this for you automatically). Note that cherry picking is undesirable, but a practical inevitability while practising trunk-based development (which is why it happens all over the place in the IC repo). This sort of thing sometimes leads to the same change being duplicated in multiple proposals.
I 
- I DO support the sentiment of this post
- I DO NOT support the sentiment of this post
If you read this far and are still interested in the 10,000 ICP giveaway... š
ā¦ all you need to do is
- take part in the poll above (your answer does not affect your eligibility for the giveaway),
- then post a thoughtful comment that explains why you agree/disagree with the sentiment of this post (a comment that shows that youāve read and considered this post),
- and then locate and share (editing and appending to your existing comment) the IC-OS version commit referenced by this proposal ā 123008 (Please post a working commit URL that links to the specific commit)
1 Winner will be selected randomly out of all (if any) who qualify. The winner will be selected in 36 hours from this post being published. Good luck!
You may also be interested in other giveaways Iāve hosted in the past (theyāre always intended to make a point)
Giveaway!!! QUICKā¦
) it definitely seems necessary and well worth it in the long run. Assuming it would be difficult to tackle this in one fell swoop Iām hoping that a poor manās approach might help in the meantime (labelling commits by deployment target). Hopefully thisāll also provide some data that will make planning the optimum git workflow easier.
I actually didnāt expect anyone would likely have access to that commit. I assumed it was more likely that it was never pushed and only hanging around in someoneās local repository (and eventually deleted as itās a year old).
Thank you