Using @dfinity/agent in node.js

coin_master · February 26, 2022, 4:39am

This looks promising , would that also work for the pem file? since our dfx identity is created using a pem file, or is there a way to get the seed phrase from a pem file?

atomikm · March 10, 2022, 7:26pm

also wondering this^

skilesare · April 18, 2022, 9:30pm

What am I missing here? I used this pattern in my node.js script, and the identity actually works fine. The problem is that when I try to include my actor from /src/declaration/project_name/project_name.js or /src/declaration/project_name/index.js I get the dreaded: SyntaxError: Unexpected token ‘export’ or Cannot use import statement outside a module. I can get around this by copying the export const idlFacotry…line from the file into my script and getting rid of the export. But this is really annoying to do every time. Perhaps I’ve missed some updates to the js world in the last couple of years, but I’ve tried a number of things and I either get that something isn’t compatible with commonjs or visa versa. Does the IDL get dumped somewhere that I can reference with a require(‘x’) statement?

peterparker · April 19, 2022, 5:16am

If you are using NodeJS LTS or more recent versions you can use script module files .mjs.

What I do is adding a postbuild script in package.json that automatically copy the .did.js files to .did.mjs files. Then in myscript.mjs script I can import the idlFactory as following (modules script support import)

import {idlFactory} from '..dfx/local/canisters/manager/manager.did.mjs';

Got a couple of scripts there GitHub - papyrs/ic: Backend canisters and providers of Papyrs

jzxchiang · April 19, 2022, 5:39am

Just to confirm… I want to use my default dfx identity to authenticate with my canister using the @dfinity/agent library in node…

Ideally using the PEM file in ~/.config/dfx/identity/default/identity.pem. I’m guessing from this thread that this is not possible right now? (I need to create a new identity / principal using keysmith and use that instead. Does that sound right?)

skilesare · April 19, 2022, 1:00pm

I think you can use Key Smith to produce a back up phrase for your default identity. Just run the help for key Smith and you’ll see a number of options.

skilesare · April 19, 2022, 1:03pm

If I do this I have to convert all of my other require statements to import statements correct? I feel like I tried this but one of my existing packages demanded that I use require or something like that. I’ll give this a try and report back.

peterparker · April 19, 2022, 1:47pm

Yes probably, a bit a pity but at least a one time change. Let me know if it works out

icme · May 29, 2022, 7:49am

Thanks to everyone above for the great commentary, really was nice to follow as I started troubleshooting not being able to load in my existing dfx principal via agent-js (I received the same different dfx and node principals as everyone else).

Since it did take me a bit of time to get through it all, hopefully these steps make it easier for those of you running into a similar issue (as of the time of this post).

Steps

Navigate to my .dfx identities → ~/.config/.dfx/identity
Create a new identity → mkdir local-testing; cd local-testing
Download quill since keysmith is now deprecated.
Test that quill is installed correctly → quill
Look up how to generate a key → quill generate --help
Generate a key and seed file → quill generate --pem-file identity.pem --seed-file seed.txt
Now implement Kyle’s code snippet below (copied from comment this post is replying to)

kpeacock:

const Identity = require("@dfinity/identity");
const hdkey = require("hdkey");
const fs = require("fs");

const { Secp256k1KeyIdentity } = Identity;

const bip39 = require("bip39");

const phrase = fs.readFileSync("seed.txt").toString().trim();

export const identityFromSeed = async (phrase) => {
  const seed = await bip39.mnemonicToSeed(phrase);
  const root = hdkey.fromMasterSeed(seed);
  const addrnode = root.derive("m/44'/223'/0'/0/0");

  return Secp256k1KeyIdentity.fromSecretKey(addrnode.privateKey);
};

identityFromSeed(phrase).then((identity) => {
  console.log(identity.getPrincipal().toString());
});

If you run this an have NodeJS version 17+, you will receive the following error coming from the hdkey library import → Error: error:0308010C:digital envelope routines::unsupported. The possible solutions for getting around this are here, the easiest of which is downgrading to node 16. (@kpeacock/sdk team can you provide a node 17+ solution? )
Now running the above node script and dfx identity get-principal should return the same identity! Yay!

lastmjs · July 22, 2022, 7:58pm

So there’s no way to just pass in a path to a PEM file and get an identity that can be used in @dfinity/agent? It used to be the case that I could just pass in JSON object and all of this worked, that was last Summer. What happened?

kpeacock · July 22, 2022, 8:45pm

a JSON object has always worked and still does work. The complicated flow is specifically for deriving an identical principal across dfx, quill, and agent-js

lastmjs · July 22, 2022, 8:50pm

Do you know where the documentation is on what the JSON object should look like? I keep getting errors with it

kpeacock · July 22, 2022, 8:53pm

github.com

dfinity/agent-js/blob/05e12ec/packages/identity/src/identity/secp256k1.ts#L12


      
          import { DerEncodedPublicKey, KeyPair, Signature } from '@dfinity/agent';
          import Secp256k1 from 'secp256k1';
          import { sha256 } from 'js-sha256';
          import { randomBytes } from 'tweetnacl';
          import { PublicKey, SignIdentity } from '@dfinity/agent';
          import { fromHexString, toHexString } from '../buffer';
          import { SECP256K1_OID, unwrapDER, wrapDER } from './der';
          
          
declare type PublicKeyHex = string;
          declare type SecretKeyHex = string;
          export declare type JsonableSecp256k1Identity = [PublicKeyHex, SecretKeyHex];
          
          
export class Secp256k1PublicKey implements PublicKey {
            /**
             * Construct Secp256k1PublicKey from an existing PublicKey
             * @param {PublicKey} key
             * @returns {Secp256k1PublicKey} Instance of Secp256k1PublicKey
             */
            public static from(key: PublicKey): Secp256k1PublicKey {
              return this.fromDer(key.toDer());
            }

ZenVoich · July 29, 2022, 6:52am

Here is working code:

github.com

ZenVoich/mops/blob/4bf6487deb7e4d3b83eabcd2c7323d9eb231e5a5/cli/pem.js

import fs from 'fs';
import {Ed25519KeyIdentity} from '@dfinity/identity';
import pemfile from 'pem-file';

export function decodeFile(file) {
	const rawKey = fs.readFileSync(file).toString();
	return decode(rawKey);
}

export function decode(rawKey) {
	var buf = pemfile.decode(rawKey);
	if (buf.length != 85) {
		throw 'expecting byte length 85 but got ' + buf.length;
	}
	let secretKey = Buffer.concat([buf.slice(16, 48), buf.slice(53, 85)]);
	return Ed25519KeyIdentity.fromSecretKey(secretKey);
}

lastmjs · August 1, 2022, 9:31pm

It does work! Thank you!

hilljh82 · September 13, 2022, 1:37am

For all those who are interested on this thread, I have created a dfinity module in BlueprintJS called blueprint-dfinity. The goal of this module is to make it easier to write NodeJS clients for the Internet Computer.

Some of the key features of this BlueprintJS module are:

Declarative approach for defining actors
Ability to bind actors and their source canisters to properties
Codifies bootstrapping code

The current version of trunk uses the original property definition approach to defining actors, and binding them to object properties. The newest version of the blueprint-dfinity, which is to be released with v5, uses decorators to define the actors and bindings to properties on ES6 classes.

blueprint/packages/blueprint-dfinity at v5 · onehilltech/blueprint · GitHub

Please have a look and provide feedback.

Disclaimer. I am the creator of BlueprintJS.

icme · December 29, 2022, 7:43pm

Heads up to anyone who’s still viewing this thread:

In September the sdk (DFX) changed from generating Ed25519 identities to Secp256k1 identities.

feat: secp256k1 keys (#2499) · dfinity/sdk@4f10e12 · GitHub, which was included in 0.12.0-beta.2, and then eventually released with 0.12.0, (see the documentation site release notes)

For example, if I go back to DFX < 0.12.0, generate an identity with dfx identity new <alias>, this solution will work, but if one uses any newer versions of dfx that uses different key generation, it will break.

@AdamS @kpeacock Are there any node based solutions in agent-js or elsewhere for pulling in the new Secp256k1 identities from a dfx >=0.12.0 generated pem file?

Also I tried playing with the blueprint solution provided by the post directly above this one , but upon testing it with identities generated by dfx >= 0.12.0 I’ve found @hilljh82’s solution now produces a different identity (maybe it worked previously but there were a few changes made?).

paulyoung · December 29, 2022, 11:16pm

In case this helps, when I was trying to verify public keys they had to be converted to DER format first.

github.com/dfinity/agent-rs

Unable to verify a caller based on a public key

opened 04:58PM - 08 Dec 22 UTC

closed 09:19PM - 08 Dec 22 UTC

paulyoung

I'm struggling to verify a caller based on a public key. I generated a private k…ey with `openssl ecparam -name secp256k1 -genkey -noout -out identity.pem` and a public key with `openssl ec -in secp256k1.pem -pubout -out public.pem` I create an identity with `let identity = Secp256k1Identity::from_pem_file(private_key_path)?;` which gives me a principal of `tvfic-z7h73-2yknr-uyp26-pru5d-bjfbs-3tdnz-mctcw-mkj5d-uecaw-aae` Based on the source code for `Secp256k1Identity` it looks like I need to provide a der-encoded public key to `Principal::self_authenticating`: https://github.com/dfinity/agent-rs/blob/391f38edc4c856660f052e3e29dff486367c1275/ic-agent/src/identity/secp256k1.rs#L71-L75 I'd prefer to support a workflow where users can copy and paste the contents of the public key PEM file, and based on how `Secp256kIdentity` is encoding the public key I found the following which seems to be exactly what I need: https://github.com/RustCrypto/elliptic-curves/blob/078363da500b04a031bcefb41bd25363b7ce1df7/p256/tests/pkcs8.rs#L57-L63 This is what I'm doing but the public key fails to parse. ```rust use ic_agent::identity::{Identity as _, Secp256k1Identity}; use p256::pkcs8::EncodePublicKey as _; fn main() { let identity = Secp256k1Identity::from_pem_file(private_key_path)?; // This represents the caller as the canister sees it. let caller = identity.sender().map_err(|err| anyhow!(err))?; trace!("caller: {}", caller); // Below represents what I'm trying to do in a canister. let public_key = vec![ "-----BEGIN PUBLIC KEY-----", "MFYwEAYHKoZIzj0CAQYFK4EEAAoDQgAERzzyM2WgleyVhRLy9UpXBg8UZkvRNCTY", "E7X4b0wxP8XED9VZpqKi0n+CBhh7Wgf1o/qCE64kxfzZSs9yOY95xg==", "-----END PUBLIC KEY-----", "", ] .join(""); // Fails with `Error: crypto error` let public_key = public_key.parse::<p256::PublicKey>()?; let public_key = public_key.to_public_key_der()?; let principal_from_public_key = Principal::self_authenticating(public_key); trace!("principal from public key: {}", principal_from_public_key,); trace!( "caller matches principal from public key: {}", caller == principal_from_public_key, ); } ``` Lastly, I was quite confused about `Secp256k1Identity` vs `BasicIdentity`. `BasicIdentity` supports ED25519 so I tried to use that first, but I couldn't figure it out. `openssl` on my machine didn't support that algorithm which I thought presented a barrier for people, and even then you'd get a v1 key which doesn't guarantee that the private key matches the public key. Curious if ED25519 is still the better option.

ZenVoich · December 30, 2022, 7:30am

Here is the code with both Ed25519 and Secp256k1 identities support mops/pem.js at master · ZenVoich/mops · GitHub (the previous link was to an old commit)

kpeacock · January 4, 2023, 5:38pm

This is my current recommendation - use a seed phrase in node.js and dfx. Internet Computer Content Validation Bootstrap

You can drop the seed phrase in seed.txt (which should always be gitignored), and the import it with dfx identity import --seed-file ./seed.txt

Topic		Replies	Views
[JS] JavaScript Agent version 0.9.0 Language Support	2	405	June 4, 2021
Http Agent Example JavaScript	0	1518	February 26, 2022
[JS] @dfinity/agent version 0.7.1 Language Support	31	2852	September 11, 2021
Issue with "@dfinity/agent" npm package: AgentError: Server returned an error: Code: 403 () Developers	3	76	July 15, 2024
Dfinity/agent package loaders error Developers	1	710	March 14, 2023

Using @dfinity/agent in node.js

Steps

Related topics