Request for help with local deploy of vetkey canister

Dear friends,

My name is Artur and I’m an IC dev and currently building a project where I would like to incorporate the vetkey encryption. I have looked at both the Shipstone’s and at Kristoferlund’s repos to try and start the development locally but whenever I build, deploy locally and try to open the canister via the Candid interface it says that it fails to find the .did file for the vetkey canister even though I clearly labeled it in my dfx.json which I also attached below.

Our project repo: GitHub - jakepeg/canister_app at feature/vetkeys

I also attached the full error at the bottom of the post.

I would really appreciate any help regarding this

{
  "dfx": "0.17.0",
  "canisters": {
    "vetkd_system_api": {
      "candid": "./vetkeys/chainkey_testing_canister.did",
      "type": "custom",
      "wasm": "./vetkeys/chainkey_testing_canister.wasm",
      "declarations": {
        "output": "src/declarations/vetkd_system_api"
      }
    },
    "backend": {
      "type": "rust",
      "package": "backend",
      "candid": "./backend/service.did",
      "declarations": {
        "node_compatibility": true
      }
    },
    "frontend": {
      "build": "pnpm --filter frontend run build",
      "dependencies": ["backend", "internet_identity"],
      "source": ["frontend/build/"],
      "type": "assets"
    },
    "internet_identity": {
      "type": "custom",
      "candid": "https://github.com/dfinity/internet-identity/releases/latest/download/internet_identity.did",
      "wasm": "https://github.com/dfinity/internet-identity/releases/latest/download/internet_identity_dev.wasm.gz",
      "shrink": false,
      "remote": {
        "candid": "internet_identity.did",
        "id": {
          "ic": "rdmx6-jaaaa-aaaaa-aaadq-cai"
        }
      }
    }
  },
  "defaults": {
    "build": {
      "packtool": ""
    }
  },
  "output_env_file": ".env",
  "networks": {
    "staging": {
      "providers": ["https://icp0.io"],
      "type": "persistent"
    }
  },
  "version": 1
}

asm.js type error: expecting argument type declaration for 'A' of the form 'arg = arg|0' or 'arg = +arg' or 'arg = fround(arg)' index.js:2:225413
Expected to find result for path candid, but instead found nothing. index.js:2:1098
Error: Call failed:
  Canister: br5f7-7uaaa-aaaaa-qaaca-cai
  Method: __get_candid_interface_tmp_hack (query)
  "Status": "rejected"
  "Code": "DestinationInvalid"
  "Message": "IC0302: Canister br5f7-7uaaa-aaaaa-qaaca-cai has no query method '__get_candid_interface_tmp_hack'"
    r http://127.0.0.1:4943/index.js:2
    bA http://127.0.0.1:4943/index.js:2
    vA http://127.0.0.1:4943/index.js:2
    n http://127.0.0.1:4943/index.js:2
index.js:2:314532
Uncaught (in promise) Error: Cannot fetch candid file
    fetchActor http://127.0.0.1:4943/index.js:2
index.js:2:314563
Error: Could not establish connection. Receiving end does not exist. trigger-autofill-script-injection.js:5:20
Source map error: Error: URL constructor:  is not a valid URL.
Stack in the worker:resolveSourceMapURL@resource://devtools/client/shared/source-map-loader/utils/fetchSourceMap.js:56:22
getOriginalURLs@resource://devtools/client/shared/source-map-loader/source-map.js:73:24
workerHandler/</<@resource://devtools/client/shared/worker-utils.js:115:52
workerHandler/<@resource://devtools/client/shared/worker-utils.js:113:13

Resource URL: wasm:http://127.0.0.1:4943/index.js%20line%202%20%3E%20WebAssembly.instantiate
Source Map URL: null

Source map error: Error: request failed with status 400
Stack in the worker:networkRequest@resource://devtools/client/shared/source-map-loader/utils/network-request.js:43:9

Resource URL: http://127.0.0.1:4943/index.js
Source Map URL: index.js.map

Source map error: request failed with status 400
Resource URL: http://127.0.0.1:4943/index.js
Source Map URL: index.js.map

asm.js type error: Asm.js optimizer disabled because no suitable wasm compiler is available
2 Likes

To me it looks like everything is working ‘as intended.’ Apparently the preview canister is not set up such that you can automatically point the Candid UI at it. For Candid UI to work without extra setup it needs to either export the __get_candid_interface_tmp_hack function or have a candid:service metadata section.

Inspecting the .wasm in your repo:

❯ ic-wasm chainkey_testing_canister.wasm info 
<...>
Exported methods: [
    "canister_update schnorr_public_key",
    "canister_update sign_with_schnorr",
    "canister_update ecdsa_public_key",
    "canister_update sign_with_ecdsa",
    "canister_init",
    "canister_post_upgrade",
    "canister_query call_counts",
    "__getrandom_custom",
    "canister_update vetkd_public_key",
    "canister_update vetkd_derive_encrypted_key",
    "canister_global_timer",
    "canister_update <ic-cdk internal> timer_executor",
]

❯ ic-wasm chainkey_testing_canister.wasm metadata
<empty>

If you go to the ‘naked’ Candid UI canister you can point it at a canister id and provide a .did file manually. If you do that then it should work

3 Likes

Perhaps the team omitted the did intentionally. The vetkey api will not be directly callable from a frontend, only from a canister.

2 Likes

Thank you both so much @Severin and @kristofer,

At least now i understand the candid ui not working is not a deal breaker for working with vetkey’s locally.

Right now I’m in the process of refactoring the project from using standard double layer encryption to using vetkeys which requires a decent amount of architectural changes.

Please ignore the part below because it’s just me trying to organize my thoughts and trying to properly architect the change to vetkeys.

Current flow:

A. Initial Setup:

  • Each user has a RSA keypair stored in their browser

B. File Upload Process:

  1. When uploading a file:
    • Generate a random AES key for the file
    • Encrypt file with AES key
    • Encrypt AES key with recipient’s public RSA key
    • Upload both encrypted file and encrypted AES key

C. File Decryption:

  1. When downloading a file:
    • Download encrypted file and encrypted AES key
    • Decrypt AES key using private RSA key
    • Use decrypted AES key to decrypt file contents

Refactored flow:

A. Initial Setup:

  • Replace keyStorage.ts with vetkeys storage system
  • Create new VetKeyService class in lib/vetkeys/encrypt.ts that will be our main interface with the vetkeys package

B. File Upload Process:

  1. In lib/components/Upload/Upload.svelte:

    • User selects file
    • System gets note ID from backend
    • File is encrypted using vetkeys’ encryptWithNoteKey
    • Encrypted file is uploaded in chunks
  2. Core encryption logic in VetKeyService:

    • Instead of generating AES keys, use IBE-based encryption
    • Each file gets a unique note ID
    • Encryption is tied to the owner’s identity

C. File Decryption:

  1. In DecryptService:
    • Download encrypted file
    • Use decryptWithNoteKey to decrypt content
    • Permissions are automatically handled by the vetkeys system

Current issue is I’m trying to use the already built @shipstone’s vetkey package and due to it having tools prebuilt for file/note encryption.

I thought it would be fitting but unfortunately I am getting more confused, could be cause I’m sick right now but also just not sure how to properly generate the noteId required for their encryption methods and how they need to relate to the actual documents (e.g. do they need to be derived in some way through hashing?).

Thinking of just doing it the same way it was done in @kristofer’s repo with vetkd.IBECiphertext.encrypt method which might just be simpler.

Btw, if anyone knows how to get in touch with the @shipstone team I would really appreciate it

Andy is pinging you now! @shipstone

1 Like

The reason for the notes infrastructure is to be able to have a place to put an access control list to be able to check before using the noteId and owner principal to derive the encryption key. Each note has metadata, encrypted data and access control lists and history. So in your case you could store the URL in metadata maybe and forgo the encrypted data for the note (if you have the data in another stable memory table for example.) Let me know where I can help.

2 Likes

Hello everyone,

Thank you again for the clarification, at the moment I decided to go with @kristofer’s implementation of the vetkey encryption (e.g. the standard one) due to it being less abstracted and thus simpler for me to navigate as a beginner in encryption.

I am currently done with most of the code and just encountering a strange issue with actually building the project.
Whenever I run dfx deploy or pnpm run build it errors saying it is failing to find the package even though I am using it in the actual code and able to reference the methods definitions and what parameters the functions take, so the situation for me is very strange.

Error: Cannot find package '/home/artur/Documents/Code/ic-docutrack/frontend/node_modules/ic-vetkd-utils/index.js' imported from /home/artur/Documents/Code/ic-docutrack/frontend/.svelte-kit/output/server/entries/pages/test/_page.svelte.js
    at legacyMainResolve (node:internal/modules/esm/resolve:204:26)
    at packageResolve (node:internal/modules/esm/resolve:778:12)
    at moduleResolve (node:internal/modules/esm/resolve:854:18)
    at defaultResolve (node:internal/modules/esm/resolve:984:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:654:12)
    at #cachedDefaultResolve (node:internal/modules/esm/loader:603:25)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:586:38)
    at ModuleLoader.getModuleJobForImport (node:internal/modules/esm/loader:242:38)
    at ModuleJob._link (node:internal/modules/esm/module_job:135:49) {
  code: 'ERR_MODULE_NOT_FOUND'
}

node:internal/event_target:1101
  process.nextTick(() => { throw err; });
                           ^
Error: 500 /test/ (linked from /test)
To suppress or handle this error, implement `handleHttpError` in https://kit.svelte.dev/docs/configuration#prerender
    at file:///home/artur/Documents/Code/ic-docutrack/node_modules/.pnpm/@sveltejs+kit@1.30.4_svelte@5.19.0_vite@4.5.5_@types+node@18.19.70_sass@1.83.1_/node_modules/@sveltejs/kit/src/core/config/options.js:212:13
    at file:///home/artur/Documents/Code/ic-docutrack/node_modules/.pnpm/@sveltejs+kit@1.30.4_svelte@5.19.0_vite@4.5.5_@types+node@18.19.70_sass@1.83.1_/node_modules/@sveltejs/kit/src/core/postbuild/prerender.js:64:25
    at save (file:///home/artur/Documents/Code/ic-docutrack/node_modules/.pnpm/@sveltejs+kit@1.30.4_svelte@5.19.0_vite@4.5.5_@types+node@18.19.70_sass@1.83.1_/node_modules/@sveltejs/kit/src/core/postbuild/prerender.js:403:4)
    at visit (file:///home/artur/Documents/Code/ic-docutrack/node_modules/.pnpm/@sveltejs+kit@1.30.4_svelte@5.19.0_vite@4.5.5_@types+node@18.19.70_sass@1.83.1_/node_modules/@sveltejs/kit/src/core/postbuild/prerender.js:236:3)

Node.js v23.3.0
 ELIFECYCLE  Command failed with exit code 1.

Looking through @shipstone’s and @richtera’s repo I noticed you guys mentioned having an issue with rollup in your vetkd_user_lib and I’m just wondering if it’s a similar issue or not.

Maybe in the future rollup will support npm packages, but as of now this doesn't seem to be the case.

My project is in Svelte and uses Vite so I’m just wondering if it’s something to do with the package currently relying on webpack or not.

Btw, I would love to comeback to try to implement the code using @shipstone’s package especially the permission stuff you have built in which seems really useful and cool.

And another thing, I sent a DM on X/Twitter to @richtera with a question I feel too silly to ask publicly cause I don’t know of another way to contact you guys, hope that’s okay.

Yes X is ok, but what’s your handle so I can recognize it; I have a bunch of unknown messages. Which project did you clone? The one project has the npm/crate inside of packages/* and the other users the public repo. Depending on which one you need either a pnpm build or pnpm add and a create add. It’s hard to tell though. I also use vite/rollup so that should be fine, but the npm does have some wasm code.

1 Like

Morning, @richtera ,

My username is Blank_JustBlank, also I’m not sure what you mean by which project I cloned since we forked ic-docutrack to build upon it but the actual vetkey part I’m writing myself.

Or did you mean I cloned your repo for vetkd-utils?

Oh I can’t find you in X form some reason. Ok so in the stack trace I saw that it couldn’t find my npm so I was trying to figure out how that could be. We have the original project and a template repo where the npm is bound as either source or from npmjs.org. If you have your own project you can use the cargo and npm modules directly. Vetkd can be used without them of course but somehow you have the dependency I think

1 Like

Hey @richtera, strange you couldn’t find me, just in case I attached a screenshot of my profile.

I managed to solve the previous issue with something I’m not sure is the correct solution, I pointed the exact path of the entry file in my vite.config.
I can now run pnpm build and dfx deploy with no issue but since I haven’t been able to make the rust code work yet I’m not 100 certain this works since I haven’t tested it.

const config: UserConfig = {
  plugins: [sveltekit(), wasm(), topLevelAwait()],
  resolve: {
    alias: {
      $lib: path.resolve("./src/frontend/src/lib"),
      "ic-vetkd-utils": path.resolve(
        "./node_modules/ic-vetkd-utils/ic_vetkd_utils.js",
      ),
    },
  },

On another note, I am struggling to figure out how to actually call methods from the vetkey_api_canister in the backend rust code, I took @kristofer 's code and just swapped one line but not sure how to actually import/use the vetkey api methods.

use crate::declarations::vetkd_system_api::{
    chainkey_testing_canister, VetkdCurve, VetkdDeriveEncryptedKeyArgs,
    VetkdDeriveEncryptedKeyArgsKeyId,
}; // this isn't working
use ic_cdk::update;
use serde_bytes::ByteBuf;

#[update]
async fn vetkd_encrypted_key(encryption_public_key: Vec<u8>) -> Result<Vec<u8>, String> {
    let address = ic_cdk::api::caller(); // Replaced ethaddress with ICP principal of the user

    let args = VetkdDeriveEncryptedKeyArgs {
        key_id: VetkdDeriveEncryptedKeyArgsKeyId {
            name: "insecure_test_key_1".to_string(),
            curve: VetkdCurve::Bls12381G2,
        },
        derivation_path: vec![],
        derivation_id: ByteBuf::from(*address.0),
        encryption_public_key: ByteBuf::from(encryption_public_key),
    };

    let (result,) = chainkey_testing_canister
        .vetkd_derive_encrypted_key(args)
        .await
        .unwrap();

    Ok(result.encrypted_key.to_vec())
}

Also if anyone could explain what the .vetkd_derive_encrypted_key method does, I would greatly appreciate it.

The vetkd container api is automatically called by the npm when you refresh the notes encryption keys. The keys are then stored in indexeddb for the notes you have access to. There are other ways, but you need to somehow whitelist which principal is allowed to use which derivation vector, because otherwise anyone can retrieve a key for any note or object.

1 Like

Hello everyone, I managed to make the file encryption/decryption work last week and wanted to thank everyone for contributing to my learning this new tech.

At the moment I am working on making file sharing working with vetkey’s and that requires from my understanding modifying what is used for derivation_id.
In my current implementation I am using the user’s principal as the derivation_id but in the case of sharing I watched somewhere you could use the principal of the user you are sharing the file for, or combining your’s and their principal.

I saw that @shipstone and @richtera package has a built in permission system and was wondering what approach you guys took.

Also @kristofer, if you have any thoughts I would appreciate it.

For anyone interested the current beta version is deployed with this url: https://4gcg2-6yaaa-aaaah-qp4pq-cai.icp0.io

Thank you everyone for being a great help and being so open with your knowledge and advice

1 Like

We didn’t change the derivation key. We use the access list to check whether the user is allowed to get the key but then derive the key by node id and owner principal. Ie the decryption key needs to be the same as the encryption key anyway. Currently you would need to re-upload to change encryption key anyways