Developer Experience, January 2021

[stanley.jones]

Hi there. I’m the Director of Engineering for Developer Experience (DX) at DFINITY. I have the privilege of managing the SDK and Languages teams. Over the last couple years they have brought you DFX, Motoko, Candid, Vessel, LinkedUp, CanCan, the Fundamentals video series, and lots of other goodies.

I’d like to talk about what we accomplished in 2020 and what we’re planning for 2021.

This post is my first on the forum, despite being one of the first people to join back in fall of 2019 when we released Motoko and the SDK. I wish I had a better excuse but, in short, we’ve been busy. We had a goal to launch the Internet Computer in 2020 and that takes a lot of work.

If you haven’t heard, we met that goal and launched the Alpha version of the Internet Computer (IC) main network on December 18th. It was the final result of our series of “metal” milestones, beginning with Copper and the launch of our SDK, though Bronze, Tungsten, Sodium, and ending with Mercury, where we are today. Our President and Chief Scientist, Dominic Williams, has written about both the launch and where the IC is headed over the next two decades here. It’s a long read but inspiring. A lot of what Dom talks about is closer than you’d guess.

In 2021, the first focus of DX is going to be feedback.

We want to hear more from you. We want to hear about what we’ve already built, what you need us to build next, and how we can pass the torch to the community so that there are more people building the IC ecosystem outside DFINITY than inside.

So far, we’ve gotten feedback from you on this forum, during workshops, as GitHub issues for our public projects, and over email. I’d like to thank all of the engineers across DX and all of DFINITY who, in addition to their day job of building the IC, also took the time to meet with you, make thoughtful posts, and answer your questions. We’ve also had terrific community admins who’ve done the same and we are all grateful for the guidance and support of Alexa. They will all still be around, but we want to take things further.

We have some new hires joining the team who have developer support and developer advocacy as their primary roles. I’ll introduce them soon but I’m telling you now because I want you to know that this commitment to feedback goes beyond words.

Finally, while this post might be my first in over a year it won’t be my last. I hope to give you consistent updates on the high-level strategy around DX at DFINITY so you can be part of the next generation of software development and help direct the growth of the Internet Computer from its first days to the beacon of decentralization and democratization we all believe it can be.

[imhello]

My habit is to mark before watching

[PeterBorah]

Hey Stanley, thanks for the post!

I’m a blockchain dev who’s been following DFINITY (off and on) for 5+ years now, and I still feel like I don’t have enough information about it to make informed decisions about what sorts of projects it’s suitable for, or to get started building anything beyond a demo. The SDK and examples are great for a quick demo project, but there’s a lot of “magic” that makes it hard to tell what’s actually going on.

Some examples of nuts-and-bolts information I’d love to have:

How are transaction fees calculated, and at what point in the transaction lifecycle are they applied? What kind of digital signature algorithm are users meant to use, and what’s the transaction format? What sort of node can accept new transactions, and is there a discovery algorithm for such nodes? What’s the message format for cross-canister calls, and what sort of properties (execution guarantees, time delays, etc) do they have? What’s the consensus algorithm and what are its properties (in terms of timing, capacity, security assumptions, etc.)? What’s the security model for canisters, and what sorts of attacks do we need to handle on the developer side?

With some basic information like that, it’ll be a lot easier to evaluate the IC and build tools to work with it.

[RMCS]

Hi Stanley, great post!

First of all i’m really looking forward to develop with this new tech. I had a question regarding developer adoption.

It would be great to install a package and use it out of the box with a existing tech like NestJS or .NET core backend frameworks.

As an example, the TypeORM package let you quickly create database schema’s, make migrations and connect to your database.

If there would be a package like that for dfinity it would be a great addition, this tackles the need to make developers learn a new language and would boost adoption.

I do not know if this would be technically possible (at the time).

Looking forward to your response,

Kind regards,
Remco

[stanley.jones]

Great questions! I’ll do my best. I’ll try to map the terms that we use at DFINITY to what you’re asking so that our documentation is a little clearer.

How are transaction fees calculated, and at what point in the transaction lifecycle are they applied?

We call your software, running on the Internet Computer, a “canister.” This term includes both your code and its state. Your canister will maintain a balance of “cycles” that it uses to pay for everything it does. Generally speaking there are three categories: messaging, execution, and storage. Sending a message to another canister costs cycles, so does executing a message from another canister or a user, and finally storing the data within your canister costs cycles. The first two are applied upon success and the last is applied at a periodic rate.

What kind of digital signature algorithm are users meant to use, and what’s the transaction format?

I could dig into the details here, but the short answer is that any application running outside the Internet Computer should probably use what we call an “agent”. The agent handles serialization to and from the IC as well as signing the message based on the identity of the user. Our Rust Agent is currently public and it’s my hope that we see folks building agents for Python, Swift, even PHP over time.

What sort of node can accept new transactions, and is there a discovery algorithm for such nodes?

There will be some nodes reserved for the Network Nervous System (NNS) but all other nodes are available for developers to run their software — and developers are encouraged to treat any of these nodes as interchangeable. The current discovery mechanism is scaffolded as something that resembles a load balancer, but we expect this behavior to eventually move into the agent (see above).

What’s the message format for cross-canister calls, and what sort of properties (execution guarantees, time delays, etc) do they have?

As a developer, the best way to think of message format is that you’re calling a method exposed by the other canister. If that method accepts two arguments, then that is what is sent (along with some metadata like the address of the target canister and an identifier of the caller).

What’s the consensus algorithm and what are its properties (in terms of timing, capacity, security assumptions, etc.)?

Not my area of expertise so I’ll reluctantly skip this one. I can say that we typically see “updates” (calls that modify a canister’s state) resolve in a handful of seconds on top of execution time and “queries” (calls that don’t modify a canister’s state) resolve in milliseconds, again on top of any execution time.

What’s the security model for canisters, and what sorts of attacks do we need to handle on the developer side?

There’s a lot of ground to cover in your question, so I’ll assume that if you’re building a web app that connects to the Internet Computer as its background you can refer to resources already out there for securing your front-end. So, as a canister developer, the most salient point in my opinion is that every call your canister will be handling is authenticated and you’ll have access to that user’s (or canister’s!) “principal” identifier. This fact enables lots of different authorization schemes, the simplest being a lookup to see if that principal has permissions to execute the action they’re requesting. We’re deliberately avoiding introducing more “automagic” security in this area until we see more of what our developers come up with.

To answer your meta-question of “for what sorts of projects is the IC suitable?” we like to say that it’s a general purpose compute platform and that, ideally, it’s suitable for any project. Now, in 2021 it’s clearly a bad choice for lots of classes of software that require low latency for updates. I would never try to run Fortnite on the IC. But I personally have been surprised at what we’ve seen already that have gone beyond mere web apps: multiplayer games, video streaming, compilers, machine learning, data pipelines… It’s more “general purpose” than you’d guess.

[stanley.jones]

I agree that would be great.

One of the constant challenges we face in Developer Experience is striking the balance between novelty and familiarity. We want to build things that work with existing tools — but we also want to evolve beyond existing tools and their limitations.

We also don’t want DFINITY’s attention to a particular technology to influence the community’s perspective on what is or isn’t possible. For example, if we built a template for Create React App then you might think the best way to build a front-end for the Internet Computer is via React. That’s exactly where our developer community comes in and there is indeed a community created template for Create React App. Thanks @Tbd!

All of which is a very long way to say that I hope you look into what it would take to build some of the things that you’d like to see for the Internet Computer and let us know how we can help.

[PeterBorah]

Hi Stanley, thanks for the long response. Unfortunately, it doesn’t really move the needle for the sort of thing I’m talking about. There were a couple details I hadn’t managed to glean from the documentation, but mostly your response is working at the same high level of abstraction as the documentation. I’m interested to know how any of this actually works, below the high-level abstractions.

It’s not possible to reason about the technical tradeoffs given the level of detail currently available. Imagine if you were trying to build an business on AWS, and their description of their fee structure was nothing more than “You pay for computation and data storage.”

[stanley.jones]

That’s fair criticism. We’ve had a strong “don’t speculate on economics” rule in engineering for the past year but I can see how some hard numbers would be valuable. Let me talk to some folks and see what I’m able to say.

[pisekchaiold]

Now react is supported, will it continue to support vue?

[Ori]

You could use any framework, the current documentation uses React since it’s a popular option but it’ll be expanded to include others, there are also some good community built examples, including Vue:
https://forum.dfinity.org/t/serving-custom-vue-frontend/1129/5?u=ori

When your web interface is first loaded from the canister the browser is served a bootstrapped html page that contains an “app” element. Whichever framework you use you’ll want to populate this element, in React it’s the line render(<MyApp />, document.getElementById('app')); but most frameworks will do something similar.

[brson]

@stanley.jones I’ve published an experience report that you will probably find useful:

https://brson.github.io/2021/01/30/dfinity-impressions

[cryptoschindler]

This is very valuable, thank you for taking the time writing this down. A few answers to the questions you posed:

• the network is permissionless and anyone can join. you would need to apply for a datecenter ID though at the network nervous system. this is an on chain governance mechanism were people can vote for your application.
• dfx --help is pretty nice. you can use dfx canister id to get the id of a canister you installed.

[Fulco]

Hey Stanley

I’ve collected a list of things that I thought were missing when developing in motoko as opposed to java and javascript which I use in my day job. I get that a lot of these thing might not have the highest priority right now but they certainly fall into the bucket of DX:

• When trying to use a type from the standard library that isn’t imported yet automatically suggest an import which can be added with a simple key press.
• At first I was confused why Text.size() gave me an unbound variable exception I had already used the Text type earlier and it was highlighted in green in my editor, but apparently you can use the type but to use the functions that come with it you need to import “mo:base/Text”. Again import suggestions would help.
• No possibility of go to definition yet in the editor.
• Its great that you already have function autocomplete when you type e.g. Trie. What would be even nicer is if it would also display the type signature and documentation like with typescript.
• On that note It would probably be good if there was an official documentation scheme for motoko
• perhaps highlight null in the extension so that it stands out more as a special keyword
• From reading the documentation it is not immediately clear where there is syntactic sugar and where not. apparently you can do text.size() but not text.trim(#text " ") it would be great to have a little example in the form of:
  let text = "test ";
let trimmedText: Text = Text.trim(test, #text " ")
  under every function in the documentation. This would have also cleared up how to use for example the Pattern variant type which might not be immediately clear to to everyone.
• I also really hope the same sugar as with text.size() would be added to the other text functions so you can do something in the form of
  text.trim(#text " ").endsWith(#text "end"). (that also goes for lots of other types btw)
• as much intellisense as possible.
• error messages could be more helpful/structured perhaps highlight the different Type letters and where there is a mismatch. Also what do the numbers here <K, V>(Trie/558<K, V>, Key/558<K>, mean?
• The ability to refactor variables or functions at all places they are being used would be nice.
• maybe hovering over certain keywords like actor could give some documentation
• anounymous/lambda functions would be nice for example instead of needing to define
  private func key(x: Word32) : Trie.Key<Word32> {
return {hash = x; key = x};
}
  being able to able to pass something like (x) -> return {hash = x; key = x}
• Also I read that the sdk now includes telemetry, it would be nice to be able to read somewhere what exactly is send especially when you look at dfinity’s mission.
• finally functions should probably be highlighted in a different color

Anyways it is great how far the project has come already keep up the great work!

[stanley.jones]

Thank you for the fantastic write-up. The entire team has read it and have already begun to dissect it into separate tasks. Hopefully you’ll see some of them resolved soon.

[bengo]

Thank you @brson and Aimee for sharing with such helpful stream-of-consciousness notes for each step of working through the getting started experience!

[rossberg]

Thanks for the feedback, all good suggestions. A couple of comments below:

• From reading the documentation it is not immediately clear where there is syntactic sugar and where not. apparently you can do text.size() but not text.trim(#text )

Yeah, that is confusing. It is not really syntactic sugar, though. Rather, there is a kind of implicit conversion between Text values and a simple object with accessors to its data content. It does not provide any methods besides that.

• error messages could be more helpful/structured perhaps highlight the different Type letters and where there is a mismatch. Also what do the numbers here <K, V>(Trie/558<K, V>, Key/558<K>, mean?

You can mostly ignore the numbers, they simply distinguish different internal occurrences of the same type identifier. The only case where it matters is if the same type name occurs with different numbers – then it’s referring to different types. We should try to suppress the numbers in other cases.

• anounymous/lambda functions would be nice for example instead of needing to define
  private func key(x: Word32) : Trie.Key<Word32> {
return {hash = x; key = x};
}
  being able to able to pass something like (x) -> return {hash = x; key = x}

Anonymous functions exist, you can e.g. write:

Array.map(a, func(x) { x + 1 });

However, you may need to annotate argument and result types where they cannot be inferred from context:

let f = func(x : Nat) : Nat { x + 1 };

[GridVision]

This is the very first computer language that I’ve not been able to get through the 'hello world" tutorial, and I’ve been writing code for over 30 years as a professional developer. I’m at a complete loss for words, and so incredibly disappointed. Although I have stumbled on the occasional bug or glitch that didn’t have any documentation or discussion whatsoever, this time I just feel like turning it off, and avoiding the path altogether. This has been the most disappointing walk down the path of a new language that I have ever experienced.

[yrgg]

I definitely agree that some time could be save with better code editor tooling/intellisense.

I’m sure it’s in the works though, as creating an entire programming language (and all the tools) is a huge process.

I think as developers we take a lot of language features for granted, and don’t realize it takes time for this stuff to appear. For example trying to google anything related to error codes + internet computer is pretty much not going to work, though we’re used to being able to google anything we want about react or typescript and dozens of answers will pop up.

We’ll just have to be patient and build the foundation so that other developers in the future will have a lower barrier for adoption.

For instance, support InternetComputer on stack overflow would be a great step toward lowering the barrier for new developers.

link to Internet Computer stack overflow post