Efficient make rules for ICP instead of direct use of DFX, to speedup development

I am applying for a Developer Grant.

Please review my application and discuss! See DFINITY’s directions for becoming a registered reviewer here. They will be collected by DFINITY. When one week passes, DFINITY will release them and they will appear as a new section on this post.

Please review my application and discuss! If you would like to submit an official review, then please use the links below to register, see the rubric, and submit a review.

I’m looking forward to everyone’s input!

Reviewer Registration | Rubric for Evaluating | Submit a Review

MY APPLICATION:
(Update: I have some success in developing rules that recompile only needed part of the Motoko code, and only when it needs to be recompiled, unlike DFX that either recompiles everything (even when nothing needs to be recompiled) or garbles the .env file when doing partial recompilation.)

REVIEWS:

Sure, I can review your application. Please share the details, and I’ll provide feedback.

@John_steven The current draft of code is at GitHub - vporton/icp-make-rules: GNU Makefile rules for DFINITY Internet Computer - it already works for Motoko code compilation.

Unlike using dfx deploy or dfx generate, my code recompiles only changed code. It creates .deps file for this, to track which Motoko files are to be recompiled.

You can see an example of how I use my make rules in practice in this repo:

A couple things I’d like to see:

  1. Commitment to very clear documentation. Many Motoko devs have never heard or make and the dfx tools do a bad job of making things like moc, didc and other things available from the command line. A getting started in 5 minutes sections that makes sure an environment (typically macos) is all set up would be incredibly helpful.

  2. Coordination with @ZenVoich to see if your work can be easily wrapped in mops.

1 Like

I made GitHub - vporton/zondirectory2: My social network that pays to authors with referral programs compiling about 3 times faster (the exact number is missing, because I removed a part of compiled code) than with direct use of DFX. That’s remarkable.

  1. OK, I will provide clear docs for non-Make users. But only after I receive the grant.
  2. It can’t be wrapped in mops, for example, because some of my Make rules call dfx that is an upper level than mops. So, I don’t see an opportunity for such collaboration.

Dear John, I released its preliminary (but working) version at GitHub - vporton/icp-make-rules: GNU Makefile rules for DFINITY Internet Computer

I introduced (as Git submodules) its real example in GitHub - vporton/NacDB: Multiple small databases for Internet Computer and GitHub - vporton/zondirectory2: My social network that pays to authors with referral programs (in zondirectory2 the speed increased about 2-3 times compared to my old Make rules that used DFX directly).

The main bug is copying .did files around. Not sure it something better can be done about this.

So, John, please review my application.

Is there work that can be done to improve the DFX work instead.
I think there should be alternatives, but if developers are going to use it it has to be easy to get, easy to use and most importantly maintained.
Im having a problem with dfx speed with compilation too but Im not sure if I would switch to something else, just for compilation and rely on that going forward
Where do you see this going? or is it a one time thing?

1 Like

Of course, I considered this variant. But given the complexity of DFX and how much it’s “screwed” regarding long compilation times, I decided to go easier route of Make rules.

I also found myself using dfx command in my make rules. So, formally, we don’t leave DFX, I just write a Make rules wrapper around it.

The rules are mostly ready (however have some bugs that I don’t know how to fix). The bug show external effects only on removal a canister from compilation list (I think, my software has the same bug as plain dfx would be also have a small bug on it, not deleting old compilation results).
I will keep maintaining these make rules, because now several my other projects depend on it.

And if it a “now” way to speedup compilation 3 times without rewriting of DFX, then I should receive the grant. Please, vote/say for me.

@qwertytrewq Thank you for the application, and @skilesare and @Gekctek for the reviews.

The grants committee agrees with @Gekctek that the scope of work does not justify a 5k grant. There has also been little evidence that there is high demand from the developer community for this tooling.

If we see more evidence for demand, we can reconsider an application with a more extended scope.