@cryptoschindler Can I just check again - should we start using the October tab on Oct 1st or Oct 15th? Also, in either case, you mentioned basing it on CET but can I suggest UTC as that’s how times appear in the proposals?
4 Likes
Please use the October tab from October 15th.
Also, in either case, you mentioned basing it on CET but can I suggest UTC as that’s how times appear in the proposals?
That’s a good suggestion. If no one has any concerns, let’s do it like this instead.
4 Likes
Thanks for clarifying that. I’ve added the new SM proposal to the October tab pending cut/paste at some point. I’m just on my phone so I couldn’t get links working properly. I’ll try and do a tidy up on it when I’m home later if no one’s got to it before then.
4 Likes
I have moved the PCM and SM topics to the September tab along with their reviews since this are the ones that I’m participating and have been posted on the October tab. Proposals 133309 and 133310 of the IC-OS topic have also been moved to the September tab by someone but not all reviews were moved with it. I would say each reviewer should move their reviews.
2 Likes
Yes, please don’t modify other participants rows if they don’t explicitly request you to do so.
2 Likes
Whoops, duly noted starting from now. Took the liberty to migrate @zenithcode team back to Sept since they were the only ones left. Sorry. Reminder @zenithcode we will start using the Oct tab starting the 15th.
@ZackDS Thank you so much for migrating the team back to september tab.
1 Like
Friendly reminder that the first review period ended and we expect your applications to the Developer Awards no more than ten days after the 15th of the month.
To refresh your memories, please read up on the details here.
7 Likes
Hello everyone,
I’m happy to inform you that we have finished reviewing the submissions for the first review period, and all participants have qualified and will receive their respective grants in due time.
While reviewing the submissions, we noticed a few things we would like to see improved in the upcoming review periods. These mostly revolve around this section in the initial post of this topic.
Missing proposal referrals
For some of the reviews, it was not immediately clear which proposal they referred to. Please always refer to the proposal you are reviewing, ideally in the headline of your post.
Reviews in the wrong topic
Please avoid creating your own topic when reviewing a proposal. Instead, reply to the topic created by the submitter. For the Protocol Canister Management, Subnet Management, and Participant Management & Node Admin topics, we are working on internal improvements to streamline the creation of topics for each proposal or for batched proposals submitted on the same day. In the meantime, please reply to the post created by the submitter, even if it isn’t in its own dedicated topic.
Missing summaries
Most reviews were missing summaries of the proposal. We would like you to, at the very least, confirm that the summary provided by DFINITY matches the content of the proposal and the changes it introduces.
Lack of in-depth reviews of commits for Protocol Canister Management & IC OS Version Election
Some reviews lacked in-depth analysis of the commits included in the proposal. We ask you to at least provide a summary for every commit, comparing what is described in the commit message with what you observe at a high level. Additionally, considering the 15-hour-per-week budget, we encourage you to conduct as many in-depth reviews as possible, verifying that the proposed changes make sense and that the implementation is sound. Please provide notes in you review showing that this work was done.
Only link to your reviews
When providing a link to a review in the table, please only link to your own reviews. If, for some reason, no review was possible or necessary for a specific proposal, please still create your own post—mentioning the proposal ID—and explain why no review was necessary or possible.
Missing reasoning for approval or rejection
Please always provide a reason when approving or rejecting a proposal.
Convoluted reviews
Please ensure your reviews are well-structured and clean, without unnecessary preambles or distracting content.
To summarize, I’d like to ask everyone to revisit the “What to do to get the grants” section and adjust their reviews accordingly.
While this may seem like a lot of critique, I’d like to mention that we are very happy with the overall quality of the reviews. Your work has already sparked numerous debates and helped identify errors.
Keep up the great work!
9 Likes
Thanks @cryptoschindler, this is useful feedback to have.
Regarding IC OS reviews in particular, are you able to point to any best practise examples, or reviews which do/don’t display the kind of standard that you’re looking for? I think examples would provide a useful learning tool.
2 Likes
Hello @cryptoschindler I have a few questions.
- Who reviews the grant application
- Who funds the grant treasury
- How do we know that the system is not being gamed considering the track record of grants in this ecosystem
Some examples and notes of what went right and what could be improved:
- proposal summary is missing
- proposal referral is missing
- structure could be improved, ideally we have
Proposal #x
Vote: …
Reason: …Reviews
[bbb8a51524]: … - great in-depth review
- reason for approval/rejection is given
- proposal summary is missing
- great in-depth review
- structure could be improved a bit, see above
- reason for approval/rejection not clear
I hope that helps.
6 Likes
Hey,
The grants committee which consists of DFINITY employees
DFINITY
Can you elaborate on the track record, I don’t understand what you’re referring to. If you want to verify grant recipients really do their “job”, you can check this table. The process is designed to be transparent.
3 Likes
What’s an example of a good proposal summary?
2 Likes
Thank you for the feedback. I believe this helps guide us in the right direction to improve our reviews. We want to make sure we’re clear about our thought process when going through each commit and the proposal as a whole. It’s also important that we explain why we voted a certain way. Typically, we’re more vocal when things go wrong, but we should also provide explanations when we approve the proposal.
3 Likes
No sorry. I will not be elaborating it, again. Its a waste of time.
1 Like
IIRC it was missing for the reviews I was looking at, maybe @marc0olo saw one?
@cryptoschindler I think regarding topic IC OS Version Election we agreed that a summary per commit makes most sense. independent from the structure of the post, I think @hpeebles did a good job here where the summary is included in his notes for each commit: Proposal to elect new release rc--2024-09-26_01-31 - #5 by hpeebles
what would be awesome for the community, if the commits would actually link to the github repo, so when reading a review they could also quickly look up the commit itself. I think this can easily be copied from the original summary of the DRE-Team.
for the subnet management topic there is actually a good example of @timk11 which even refers to a post that clarified the status of a node: Subnet Management - 6pbhf (Application) - #21 by timk11
3 Likes
small addition to the IC OS Version Election topic:
- during a recent call with @lara and @cryptoschindler, I mentioned that we should maybe also have a mechanism to verify whether the provided (commit) summary within the proposal is actually accurate. I see that this seems to be an issue with the repository structure that is currently in place, as many irrelevant commits need to be omitted at this point.
- @cryptoschindler made me aware of a related thread started by @Lorimer here:
- I would personally also encourage all reviewers to check for missing commits in the proposal as I am currently not sure if the process around that is “solid” enough and whether sth. significantly changed since the discussion in the thread of @Lorimer
3 Likes
Like this? →
Other aspects to IC OS proposals that haven’t yet been mentioned include unelections, e.g. →
To my knowledge, this aspect is currently not a feature of other reviews. Issues here have caused proposal execution failures in the past.
Are you able to clarify what you mean (is this something that other reviews could improve on, such as the other reviews for that same proposal)?
I rejected 5 IC OS proposals last month due to inaccurate proposal summaries (regarding which commits were included). Even the 2 proposals that I adopted had inaccuracies (though less significant ones, which is why I adopted).
This is still the case, but it’s slowly getting better. We need reviewers to be putting the pressure on - not just me. I personally think there’s a long way to go though (the issues and recommendations described in the ‘Do U C What IC’ article are still just as relevant today).
1 Like