Using @dfinity/agent in node.js

Thanks to everyone above for the great commentary, really was nice to follow as I started troubleshooting not being able to load in my existing dfx principal via agent-js (I received the same different dfx and node principals as everyone else).

Since it did take me a bit of time to get through it all, hopefully these steps make it easier for those of you running into a similar issue (as of the time of this post).

Steps

  1. Navigate to my .dfx identities → ~/.config/.dfx/identity
  2. Create a new identity → mkdir local-testing; cd local-testing
  3. Download quill since keysmith is now deprecated.
  4. Test that quill is installed correctly → quill
  5. Look up how to generate a key → quill generate --help
  6. Generate a key and seed file → quill generate --pem-file identity.pem --seed-file seed.txt
  7. Now implement Kyle’s code snippet below (copied from comment this post is replying to)
  1. If you run this an have NodeJS version 17+, you will receive the following error coming from the hdkey library import → Error: error:0308010C:digital envelope routines::unsupported. The possible solutions for getting around this are here, the easiest of which is downgrading to node 16. (@kpeacock/sdk team can you provide a node 17+ solution? :slightly_smiling_face: )
  2. Now running the above node script and dfx identity get-principal should return the same identity! Yay!
4 Likes

So there’s no way to just pass in a path to a PEM file and get an identity that can be used in @dfinity/agent? It used to be the case that I could just pass in JSON object and all of this worked, that was last Summer. What happened?

a JSON object has always worked and still does work. The complicated flow is specifically for deriving an identical principal across dfx, quill, and agent-js

Do you know where the documentation is on what the JSON object should look like? I keep getting errors with it

Here is working code:

6 Likes

It does work! Thank you!

1 Like

For all those who are interested on this thread, I have created a dfinity module in BlueprintJS called blueprint-dfinity. The goal of this module is to make it easier to write NodeJS clients for the Internet Computer.

Some of the key features of this BlueprintJS module are:

  • Declarative approach for defining actors
  • Ability to bind actors and their source canisters to properties
  • Codifies bootstrapping code

The current version of trunk uses the original property definition approach to defining actors, and binding them to object properties. The newest version of the blueprint-dfinity, which is to be released with v5, uses decorators to define the actors and bindings to properties on ES6 classes.

blueprint/packages/blueprint-dfinity at v5 · onehilltech/blueprint · GitHub

Please have a look and provide feedback.

Disclaimer. I am the creator of BlueprintJS.

4 Likes

Heads up to anyone who’s still viewing this thread:

In September the sdk (DFX) changed from generating Ed25519 identities to Secp256k1 identities.

feat: secp256k1 keys (#2499) · dfinity/sdk@4f10e12 · GitHub, which was included in 0.12.0-beta.2, and then eventually released with 0.12.0, (see the documentation site release notes)

For example, if I go back to DFX < 0.12.0, generate an identity with dfx identity new <alias>, this solution will work, but if one uses any newer versions of dfx that uses different key generation, it will break.

@AdamS @kpeacock Are there any node based solutions in agent-js or elsewhere for pulling in the new Secp256k1 identities from a dfx >=0.12.0 generated pem file?

Also I tried playing with the blueprint solution provided by the post directly above this one :point_up: , but upon testing it with identities generated by dfx >= 0.12.0 I’ve found @hilljh82’s solution now produces a different identity (maybe it worked previously but there were a few changes made?).

In case this helps, when I was trying to verify public keys they had to be converted to DER format first.

Here is the code with both Ed25519 and Secp256k1 identities support mops/pem.js at master · ZenVoich/mops · GitHub (the previous link was to an old commit)

2 Likes

This is my current recommendation - use a seed phrase in node.js and dfx. Internet Computer Content Validation Bootstrap

You can drop the seed phrase in seed.txt (which should always be gitignored), and the import it with dfx identity import --seed-file ./seed.txt

1 Like

@kpeacock

Is there an issue with the Ed25519 and/or Secp256k1 solutions provided by ZenVoich?

Several developers I’ve spoken with prefer for the solution that imports the dfx generated pem file instead of the quill generated seed.txt (which they’ve professed the seed.txt derived identity import feels like a “hacky”, “roundabout”, or “unstable” solution).

It’s also about consistency, mostly because then developers are using two different tools to manage identity/canisters.

there’s nothing wrong with using the PEM files - we just haven’t added official support or documentation for it yet.

instead of the quill generated seed.txt (which they’ve professed the seed.txt derived identity import feels like a “hacky”, “roundabout”, or “unstable” solution).

Quill is unnecessary, unless you’re trying to use an existing PEM file in a seed phrase context. I simply suggest using a freshly generated seed phrase from your tool of choice, which will work nicely in CI contexts

1 Like

Nice script @ZenVoich :+1:. Do you know how to request a pwd and decode the pem file when encrypted?

1 Like

@peterparker do you mean how to decrypt the dfx-encrypted PEM? You can see the process here. You will also need the nonce from your identity’s config, which is located at .config/dfx/identity/<identity name>/identity.json

1 Like

I meant decrypt the encrypted PEM file in NodeJS.

Above script of @ZenVoich works like a charm to replicate dfx identity in NodeJS when the PEM is not encrypted.

Also attempting to use an encrypted dfx identity in node. It looks like @ZenVoich’s script is attempting to perform the decryption, but I didn’t get it working in practice when I tried it just now.

Anyone else had luck with this yet? I believe performing a decryption and using the identity in memory is the preferable solution for running privileged actions in node locally.

you could probably use spawn a process to run dfx identity export if you have the right permissions

1 Like

to support the latest agent release 0.21.3 that uses noble it seems like you only need to pass in the secret key, therefore adapt decode in @ZenVoich script to the following

function decode(rawKey: string) {
	const buf: Buffer = pemfile.decode(rawKey);
	if (rawKey.includes('EC PRIVATE KEY')) {
		if (buf.length != 118) {
			throw 'expecting byte length 118 but got ' + buf.length;
		}
		return Secp256k1KeyIdentity.fromSecretKey(buf.subarray(7, 39));
	}
	if (buf.length != 85) {
		throw 'expecting byte length 85 but got ' + buf.length;
	}
	return Ed25519KeyIdentity.fromSecretKey(buf.subarray(16, 48));
}

i didnt test the secp256k1 identities, so not sure if any changes are required there

2 Likes