[RFC] tech_stack - Canister Metadata Standard extension

Developers
1

In the original forum post [RFC] Canister Metadata Standard, many community CDK authors showed interests in adding cdk:name in the Canister Metadata Standard so that tools like Canister Explorers can fetch and analyze the data.

We, the Dfinity SDK team, did receive the message. The requesting feature is about to land.

We adopted a very different design which accommodated community suggestions. Please check the final design.

Outdated design

To make this feature more versatile and future-proof, I named the new JSON object in the dfx metadata as tech_stack. The work is in this draft PR.

Example

{
  "tech_stack": {
    "rust": "1.76.0",
    "ic-cdk": "0.13.0",
    "wasm-tools": null
  }
}

As you can see, the tech stack includes but not limit to the programming languages, CDKs, libraries, tools.

dfx.json configuration

The above metadata JSON object was generated from the dfx.json configuration below.

{
  "canisters": {
    "foo": {
      "type": "custom",
      "tech_stack": [
        {
          "name": "ic-cdk",
          "version": "0.13.0"
        },
        {
          "name": "rust",
          "version_command": "rustc --version | cut -d \" \" -f 2"
        },
        {
          "name": "wasm-tools"
        }
      ]  
    }
  }
}

For each tech stack item, the canister author can:

  • set no version
  • set the version directly
  • set the version using a command

Please check the document page for more details.

1 Like
2

How would you say the name of the CDK?

How would you set the supported languages?

CDK name has no version. Languages have no version.

3

I don’t know, I would like to deliberate on this more. CDK name, languages, etc seemed pretty simple and I liked it.

1 Like
4

Just azle: null, kybra: null, typescript: null, etc? Or with versions?

5

Maybe we should call it something else? Dependencies? It almost seems like a list of dependencies.

Also what if you want to add more information than just a version?

I would love to see this be more general-purpose, can we have more arbitrary name/value pairs instead of just name and version?

What if I want to add an ICP or Ethereum address to a dependency for example? Cycle compensation is a feature I hope is incorporated into the protocol in the future where cycles are not only burned but distributed to canister dependencies, this metadata might be the best place to put the information required to get that to work.

6

It would be nice to be able to still say name: cdk, value: azle for example. Or name: languages, value: JavaScript, TypeScript

This will make it easier for those reading the metadata to grab what they are interested in, rather than having uncategorized fields

7

I can see where @lastmjs is coming from. I think that the suggestions in the original thread seem more helpful for generating stats such as these:

image
image656Ă—592 47.3 KB

With the tech_stack approach described above, it’s just an array of key-value pairs with no context as to what each one represents, making it a bit more challenging for consumers to decipher.

2 Likes
8

I just updated the design and implementation of this feature.

PTAL

cc @Dylan

P.S. The new design was inspired by the ProducersSection of the wasm standard.

1 Like
9

Feature complete

A more concise design was adopted and the PR was merged!

{
  "tech_stack": {
    "language": {
      "rust": {
        "version": "1.75.0"
      }
    },
    "cdk": {
      "ic-cdk": {
        "version": "0.13.0"
      }
    },
    "lib": {
      "ic-cdk-timers": {},
      "ic-stable-structures": {}
    },
    "other": {
      "bitcoin": {
        "address": "bcrt1qfe264m0ycx2vcqvqyhs0gpxk6tw8ug6hqeps2d"
      }
    },
    "tool": {
      "dfx": {}
    }
  }
}

Please check the documentation for more details of the final design.

State of the Art

In the same PR, I added corresponding support for canister types known by dfx.

  • For Rust and Motoko canisters, dfx sets tech_stack if they don’t define tech_stack explicitly in dfx.json .
  • For Azle and Kybra projects created with dfx new, the corresponding tech_stack configuration will be added to dfx.json by default. (cc @lastmjs )

For CDKs not yet known by dfx (e.g., @icpp), the CDK providers can add this tech_stack section in dfx.json of the project templates.

For Canister Explorers (e.g. icp.zone by @ZenVoich), it’s expected that more canisters will have a public metadata with key “dfx”, in which there is a “tech_stack” object.

What’s Next?

We don’t envision that canister developers will define these values in dfx.json by themselves.

Upcoming work will enable CDK providers to set tech_stack fields for their users.

3 Likes
10

Awesome!

Can you also define the exact details of the custom section that is added to the wasm?

I can then implement adding it by default to C++ wasm as well.

11

In this PR, I added dfx schema --for dfx-metadata which display the schema of the “dfx” metadata JSON.

The generated statical JSON schema file can be found here.

You can also refer to the dfx.json of Azle and Kybra templates.

1 Like
12

Thanks for all of this work. I do have a concern about the process of this RFC though.

It seems you at DFINITY are just making the final decision and merging PRs without waiting for the community members involved to also agree on the final standard. Can we improve this?

If it’s an RFC I would expect final decisions to involve those requested to comment more. For example the OP begins with The requesting feature is about to land. This happened without allowing community members to review the new design (at least that’s what I felt).

Then I and others came in after the fact to request changes.

So for example I would expect something like: Here’s the PR to comment on or comment here in the forum thread. We’ll leave it open until this date if there is no further deliberation, then we’ll merge it if there’s agreement. Something more like how some of the working groups have been working.

4 Likes
13

Thank you for taking the time to provide this valuable feedback. We genuinely appreciate your input and recognize the importance of involving the community in the decision-making process for RFCs. Your concerns have been duly noted, and we are committed to improving our approach to ensure greater transparency and inclusivity in the future.

Finding the right balance between speed and process integrity is crucial, and your feedback helps us to better navigate this challenge.

When I was assigned this task, I initially perceived it as less critical compared to the more serious ICRC standards, so I leaned towards prioritizing speed over a more comprehensive RFC procedure. However, I was also keen on maintaining community involvement, especially since I was designing a metadata standard intended for use by a third-party canister explorer rather than being consumed directly by dfx . Therefore, gathering input from the eventual users was crucial to ensure that the metadata would be genuinely useful.

2 Likes
14

Question: For Azle and Kybra projects created with dfx new, the corresponding tech_stack configuration will be added dfx.json by default.

Can you guide me to where the tech_stack will be created for Azle and Kybra with dfx new? I am now integrating tech_stack into the Azle and Kybra dfx extensions, and I would like to ensure that they are all universal.

In fact, it’s probably best that dfx new simply remove tech_stack for Azle and Kybra, as the dfx extension will soon be released and will cover this.

I am also curious on people’s opinions of including the version of libraries in this information, do you think it could be a security vulnerability?

15

I’m having trouble actually reading the metadata, how do I do it?

Here’s the dfx extension I am playing with:

{
    "name": "azle",
    "version": "0.21.1",
    "homepage": "https://github.com/dfinity/dfx-extensions",
    "authors": "",
    "summary": "",
    "categories": [],
    "keywords": [],
    "canister_type": {
        "defaults": {
            "candid": ".azle/{{canister_name}}/{{canister_name}}.did",
            "candid_gen": "http",
            "build": "npx azle {{canister_name}}",
            "wasm": ".azle/{{canister_name}}/{{canister_name}}.wasm",
            "gzip": true,
            "metadata": [
                {
                    "name": "candid:service",
                    "path": ".azle/{{canister_name}}/{{canister_name}}.did"
                }
            ],
            "tech_stack": {
                "cdk": {
                  "azle": {}
                },
                "language": {
                  "typescript": {},
                  "javascript": {}
                }
            }
        }
    }
}

And I’m trying to run commands like dfx canister metadata async_await techstack:cdk

16

Something else that is strange to me, why is tech_stack a stand-alone property? Why isn’t it defined inside of the already-existing metadata property?

17

Okay so it is dfx canister metadata [canister_name] dfx. This seems a very strange and unexpected place to put this. What does dfx have to do with it really besides automatically putting it in?

I was expecting dfx canister metadata [canister_name] tech_stack, and also dfx canister metadata [canister_name] tech_stack:cdk, dfx canister metadata [canister_name] tech_stack:lanuage, etc

18

So two biggest issues I see are:

  1. Why isn’t this defined inside of the metadata property that already exists?
  2. Why is it at the dfx metadata property?
19

The “tech_stack” object is a part of the existing “dfx” metadata. So you should do:

dfx canister metadata async_await dfx

Something else that is strange to me, why is tech_stack a stand-alone property? Why isn’t it defined inside of the already-existing metadata property?

“tech_stack” and “pullable” are the configurations for dfx usage which will be assembled into the “dfx” metadata.

In the “metadata” section, you define some metadata with specified key, value, visibility.

20

I’m suggesting that it makes more sense to have a tech_stack metadata top-level property, and to just define tech_stack inside of metadata. Just make it a standard metadata property, like I felt the original intention of the RFC was.

I would be interested to know why dfx makes sense as the top-level metadata property. dfx might be the tool that’s placing the metadata in the binary, but that’s not even necessary. I could use ic-wasm or edit the Wasm in another way.

I feel like these things should be revisited.