Voting Challenge Proposal

Governance
1

Voting Challenge Proposal

TL;DR

This proposal is intended to be a community lead effort to move the IC in the direction of decentralization on the Replica Version Management NNS proposal topic. It proposes crowdfunding for bounties to pay people and organizations to formally review Bless Replica Version proposal types. It is intended to attract and recognize people in the community who perform quality work on these code reviews. The program would be administered by the proposal lead, Wenzel Bartlett, and is intended to scale if proven successful. Please consider making donations to fund this program (see Crowd Fund section at the end). Formal progress reports and as well as proof of each code review will be submitted via NNS Governance Motion proposals each week. This program does not require any changes by DFINITY to the current process for issuing and voting on these proposal topics, nor does it require any changes to tokenomics to incentivize participation. The program can be a successful example of decentralization if adequate donations are received each week to attract reviewers.

The Problem

There has been recent activity in the community from individuals (see Resources section) who have considered performing technical reviews of Replica Version Management proposal topics (for example, proposal 100821 submitted today). These reviews are arguably the most important opportunity at this time that can advance the decentralization of the internet computer. However, performing these reviews properly is real work which likely will not happen consistently unless there are incentives. Genesis occurred 20 months ago and there are many well-known organizations and many ICP whales, but none have formed named neurons that people can follow and none are publicly contributing to the development or review of the IC protocol. Hence, people and organizations with natural incentives (e.g. large stakes and/or major projects) to make meaningful contributions to the decentralization of the IC don’t seem to be contributing at the protocol level. We all depend on DFINITY, which most people will readily admit is not very decentralized. However, everyone, including DFINITY, likely agrees that we need to be on a path toward decentralization that will take many years. Community code contributions need to start somewhere. Therefore, I want to start an initiative to help catalyze change that enables developers of all skill levels to receive public recognition for demonstrating their technical skills in a way the benefits decentralization of the IC. Specifically, I want to administer and scale a program designed to crowdfund bounties that will be paid to people who provide quality technical reviews of the Bless Replica Version proposal type, which falls under the new proposal topics called Replica Version Management. It won’t be a perfect system initially, but perhaps it is a good start that can evolve into something better. The goal is decentralization and developer recognition.

The Goal

In my opinion, success is the identification of people or organizations who are willing to routinely perform technical reviews for proposals that are submitted weekly in the Replica Version Management proposal topic. This topic was created 4 months ago (proposed by @Manu and implemented by DFINITY) in order to separate the Bless Replica Version and the Retire Replica Version proposal types from more frequent proposals. This provides an opportunity to engage the community in replica reviews without an overwhelming workload. The path to decentralization requires that people and organizations that perform these reviews are committed to doing it often enough and reliably enough that they can be selected as Followees by NNS participants for this proposal topic. In other words, if we want decentralization, then we need Followee options that don’t exist today. At this time, this proposal topic falls into the All Topics catch all category, which means most existing neurons were configured to follow DFINITY when the neuron was created and there has never been a reason for people to change that selection. Natural incentives have not been enough so far, so perhaps supplemental incentives can result in people and organizations that make a commitment to becoming a reliable reviewer.

The minimum definition of success is public recognition of developers who can perform this task (perhaps it will lead to job opportunities). The desired outcome is for more than 3% of total voting power in the NNS to choose to follow neurons other than DFINITY on the Replica Version Management proposal topic. A stretch goal is for 5-25% of total voting power in the NNS to follow neurons other than DFINITY on the Replica Version Management proposal topic. The exact number is somewhat arbitrary. However, it can be considered a simple measurement of the extent of decentralization on this topic. These Followee changes are 100% voluntary, low risk to the ecosystem, and will be a slow, yet high value, transition toward decentralization.

Please note that 100% of total voting power in the NNS currently follows DFINITY on this proposal topic due to default following. DFINITY directly owns 23% of this total voting power. Hence, there is no risk of the community getting it wrong by participating in these reviews and making these changes to their Followee selection. DFINITY is still able to cast their vote just like they have in the past and it will still initiate Absolute Majority even if a major fraction of NNS participants change their Followees. It would require 50% of total voting power in the NNS to change their Followee in order to introduce a risk of getting it wrong and they would all have to vote opposite DFINITY. We don’t have that much total participation on the Governance proposal topic, which is the most decentralized proposal topic so far, so getting it wrong is truly a non-issue.

Also note that DFINITY can still vote any time they want (e.g. critical updates) and people can still get their voting rewards even if they are not following DFINITY. Proposals are executed by Absolute Majority immediately after more than 50% of total voting power is cast as either Yes or No, but that does not end the voting period. The voting period stays open for at least 4 days on every proposal. Hence, if DFINITY casts their vote 2 hours after submitting a proposal and a notable fraction of the NNS voting body is following someone else, everyone will still vote when their Followee votes. The votes won’t change the result, but the vote does count toward voting rewards. The key is for NNS participants to select Followees who are known to be reliable at always voting, which means we need to provide public opportunities and incentives for people to prove their reliability on this topic.

The Proposal

Voting Challenge

Every week, the Administrator will submit a NNS Governance Motion proposal titled Voting Challenge Status Update and Crowd Fund Request. This proposal will be submitted on Friday or Saturday and will outline 1) the bounty and percent allocation that is available, 2) the specific proposal ID that is offered for bounty review, 3) an updated explanation of the Voting Challenge for the week, 4) the bounty distribution details for the past week, and 5) a summary of the successful and unsuccessful reviewers for the past week.

Every week, people and organizations who want to perform replica reviews and get paid for this work will submit individual NNS Governance Motion proposals that demonstrate the results of their review. These motion proposals must be submitted to the NNS before the end of the Voting Period for the proposal that they are reviewing. In order to qualify, each proposal must include all details outlined in the Review Requirements section below. Since this review will be submitted as a motion proposal, the NNS governing body will decide if the review that is submitted passes quality inspection based on the Accept / Reject status. Each individual / organization may submit one review. All proposals that are accepted by the NNS will be paid from the available bounty. There is a good probability that the NNS governing body will be somewhat lenient in the beginning but may develop high expectations as the Voting Challenge continues and examples of quality work are produced by reviewers. It would be ideal if the bounty were significant enough to attract talent and to give reviewers an opportunity to hone and show off their skills in a formal way.

Requirements for Review (submitted as NNS motion proposal)

  1. Reviewer name (an anonymous identity is acceptable)
  2. Reviewer NNS voting neuron ID (this can be different than the neuron ID that is used to submit the proposal and will be used to verify the vote was cast)
  3. Reviewer account ID (the account that will get paid from the bounty if the proposal is Approved)
  4. Reviewer social media username (so unknown anonymous identities can be validated)
  5. Link to social media post showing proposal id and review conclusions (part of validation)
  6. Proposal ID (corresponding to the proposal that is being reviewed)
  7. Replica Version ID (corresponding to the proposal that is being reviewed)
  8. Release Package sha256 hex result
  9. Computer screen capture showing sha256 function command, the hex result, a unique computer ID, and the computer date/time (proof of performing work)
  10. Review Summary (outline of findings from the review)
  11. Review Methodology (describe how the review was performed)
  12. Anything else that you want to submit that you believe helps give you credibility to the NNS governing body as a reviewer.

The requirements as written can possibly be gamed, so additional suggestions to help minimize that potential are welcome. The requirements can be modified to include anything reasonable that uses existing, common infrastructure for both the user and the NNS. In other words, it is not in scope to immediately develop new tools to prove this review work was performed by unique individuals/organizations. This will start as an experiment. We will learn and improve as we go including potentially developing new tools.

The NNS will be used because it offers a formalization of the process that brings heightened awareness to NNS participants. Governance motion proposals do not change code. However, all proposals that are proposed in this Voting Challenge are about IC code and the formalization of a review process intended to engage the community and improve decentralization.

Bounty Distribution

Below is an example of how the bounty that is raised through crowd funding could be distributed. This example assumes 1000 ICP is raised for bounties and the fiat price is $4/ICP. It is not known how much funding will be raised, so these numbers are only examples and will be adjusted on a weekly basis to match actual donations.

  1. Proposal Reject Fee for Status Updates: Fixed Cost = 10 ICP = $40
  2. Voting Challenge Administration: 15% = 148.5 ICP = $594
  3. Voting Challenge Bounty: 85% = 841.5 ICP = $3366

The Voting Challenge Administration allocation would be used to 1) pay the administrator for services, 2) potentially form a legal entity such as a non-profit 501(c)(3) to enable tax deductions for donations, and 3) build capital to potentially pay for developer services that may be needed to build software tools that support the Voting Challenge program. I am proposing myself (Wenzel Bartlett) to be the Administrator.

The Voting Challenge Bounty will be divided among all reviewers who receive Approval of their review by the NNS. Goals, deliverables, work process, and fund distribution may be adjusted as the program progresses. This will start as one specific bounty for one specific proposal each week but will scale if crowd funding levels and review participation is successful. If there are no reviews for a given week, then the Voting Challenge Bounty portion will carry over to the following week and will be added to any new bounty that is crowd funded. If the weekly Status Update proposal submitted by the Administrator is rejected, or a Voting Challenge Bounty cannot be raised, then that will be taken as a sign that the NNS governing body is not interested in continuing this Voting Challenge program.

Resources

Here are many examples of manual code reviews: Voting Challenge portal on DSCVR by @bjoern
Here is an example of code review automation: Taggr discussion by @christian
Here is a potential example of team review: Forum Discussion by @lastmjs

Other Notifications

@bjoernek @diegop @Jan @dominicwilliams @lomesh @paulyoung

Proposal Lead

Wenzel Bartlett
sayitkind on Twitter and Reddit; BartlettWenzel on Telegram; wpb on the forum, Taggr, DSCVR, and Distrikt
IC ecosystem credentials: Manager for Synapse.vote neuron; Founder for CrowdGov.org website
Neuron ID: 12008772471346176261 (same neuron I have used many times to submit proposals)

Crowd Fund

Account ID: 10517e297382fd13abf9a1d05ab35b67a6875e26462d881d87418206094e4b84
Donations are non-refundable and will be used to support this Voting Challenge. Weekly crowd funding is proposed in order to start small and only scale if the program proves to be successful.

14 Likes
2

Thank you @wpb for this initiative.

Quick question: What is meant by Unique Computer ID in Review Requirements #9?

Computer screen capture showing sha256 function command, the hex result, a unique computer ID, and the computer date/time (proof of performing work)

1 Like
3

Good question. This is what I had in mind…

When you build the replica on your system and then generate the hex from the sha256 function, I’m suggesting that the screen capture you provide should contain more than just the hex result. In my example below, I ran the dfx --version command and see the dfx 0.12.1 result. However, the screen capture also shows that the command was run from my root directory in Ubuntu as wpb@TABLET-54EUCPR5 as well as the computer date/time. I was intentional about making sure all that information is included in the screen capture so it can differentiate my result from any other person who generates the same result. The hex result that can be verified in this way is already in every DFINITY proposal, so my intent is for you to prove that you actually built the replica and ran the sha256 command as part of your review. There may be other ways of doing it, but the intent is to ask for unique proof of work that is easy for people to review.

By the way, this is just my current thinking on what kind of proof can be requested, but I’m open to additional suggestions or better wording.

image
image1122Ă—636 81.6 KB

2 Likes
4

I’m curious how the following scenario would be handled.

  • Person A spends several hours reviewing and finds a bug.
  • Person B spends several hours reviewing and find the same bug.
  • Person C spends no time reviewing but claims to find the same bug as person A.

I think the answer is that we don’t know, and that it comes down to how people vote.

It might discourage people from performing reviews though, since their work could either be easily copied (which reduces their rewards) or their work could be incorrectly rejected (which means no rewards at all)

One way to solve this could be by using a 2-step process where people make submissions before some deadline without them being revealed to others, and then a second phase where people vote.

4 Likes
5

I think it will be interesting to see how it plays out. My hope is that everyone takes the activity seriously and produces thorough, professional documentation that makes it easy to differentiate. The conclusion may be the same, but perhaps the methodology and documentation will be different. When multiple people start producing the same documentation, then maybe the NNS voters will become skeptical and only award approval to the early submissions. Ideally, this could be a good opportunity to gain name recognition and professional credibility, so people may do themselves more harm than good if they are trying to game the system by copying the work of others. I do expect that new tools will be needed to ensure fairness eventually, but my preference is to get the program started and see how it goes. I hope the publicity this could bring within the developer community may be an effective deterrent to cheating. There is more to gain by doing a quality review because of the high visibility of motion proposals.

6

This is probably a good place the expand on some of my current thoughts on how this Voting Challenge program may need to scale (if it can gain traction). Perhaps hiding results before a deadline is part of a good strategy, but what I’d really like to accomplish is having a way to proving uniqueness of each review in an automated way. I have no doubt new tools will need to be created and I’m under no illusion about my inability to develop them myself. I can attempt to ideate, lead, and administer a program like this, but it will require incentivizing others to get involved to develop sound solutions. I think it will require funding, which is why there is an Administration allocation in the Crowd Fund distribution of this OP. I will also be applying for developer grants and possibly even CrowdFund-NFT rounds, but that would likely be for the development of tools and website features. However, in order for there to be a meaningful decentralization of governance over time on non-Motion proposals, I believe there will need to be some sort of routine funding. Hence, I’m giving it a shot at asking for weekly donations as a start.

Within the next month I am expecting that development of the MVP for crowdgov.org will be complete. I really want the community to be involved in scoping how that app evolves. One feature that I am very interested in considering is related to automation of this Voting Challenge program in a way that makes it clear that people are performing the work of reviewing replica updates as well as automating reviewer recognition and payments for that service. For example, the app could collect information manually entered by each reviewer (e.g. their manual work product such as information listed in the Requirements section of the OP), request permission to collect device or product information from the computer that is being used to build the tar file for the replica, initiate the build, perform the sha256 function on the tar file that is produced, report the hex result as one of the review outputs, and create and submit the NNS motion proposal that shows the output file and a certification that the review and reviewer is unique. This output does not need to contain all the personal information that may be necessary to collect as long as the review can be reasonably certified, the NNS voters can make educated decisions, and reviewers can gain credibility for this work. Perhaps there would be a CrowdGov CLI component for this purpose, but perhaps the information is entered into a page on the website that is gated by some sort of KYC or other proof of unique humanity. I don’t know all the right answers at this time, but it is something that seems worth pursuing. In fact, I want to open source CrowdGov.org to help encourage the community to participate in the development of this kind of tooling, but I also want to do it in a way that is not expecting people to work for free. All of this is real work and people need to be compensated in my opinion. I think there is a real opportunity for the community to step up as contributors to the IC, to bootstrap, recognize, and promote new and existing developers, and to slowly progress toward decentralization of the IC. I know that the current iteration of this proposal has deficiencies, but I don’t want that to stop us from trying to make progress so we can learn as we go.

7

I was trying to read this when distracted by three other things, but I’m having a really hard time figuring out what exactly is going on here. Could you maybe put this post in the chat GPT and tell it to explain your proposal to a high school sophomore?

Or maybe just tell me what you want me to do?

I’m happy to help out from a ICDevs perspective here, but I’m just not quite sure how much time/energy this will take.

Also, we have our new version of Axon coming out next week that might allow you to build and administer a voting situation thing that would help simplify this.

1 Like