Using @dfinity/agent in node.js

So I’ve been trying to run queries from node.js, and wanted to document the process, if anyone is interested. I’m also hoping some of the workarounds that I found can be properly patched so we can use services like heroku without having to jump trough hoops.

:~/work/dfinity/node_test_2$ node -v
v10.19.0

:~/work/dfinity/node_test_2$ npm -v
6.14.4

:~/work/dfinity/node_test_2$ npm init

Following the readme from here, I installed the agent package.

npm i --save @dfinity/agent

I created a file app.js, with a simple import, and ran it with
node app.js

const actor = require("@dfinity/agent");

First series of errors seem to stem from the fact that npm doesn’t import other packages. I had to add them manually.

Error: Cannot find module '@dfinity/principal'
#fix:
npm i --save @dfinity/principal

Error: Cannot find module '@dfinity/candid'
#fix:
npm i --save @dfinity/candid

#will need later:
npm i --save node-fetch

With all importing errors fixed, I ran node app.js again, and got the following error:

ReferenceError: TextEncoder is not defined
    at Object.<anonymous> (~/work/dfinity/node_test_2/node_modules/@dfinity/agent/lib/cjs/auth.js:19:50)

This can be fixed by patching the auth.js file, adding the following line after use strict:

global.TextEncoder = require("util").TextEncoder;

With the file patched, we can now import the agent without any errors. Now let’s build an actor for a local deployment, and make a call to a local canister:

app.js:

const Actor = require("@dfinity/agent").Actor;
const HttpAgent = require("@dfinity/agent").HttpAgent;

//a mix of greet, and whoami capsules, with another greetq (as query) added
const idlFactory = ({ IDL }) => {
    return IDL.Service({
        'greet': IDL.Func([IDL.Text], [IDL.Text], []),
        'greetq': IDL.Func([IDL.Text], [IDL.Text], ['query']),
        'whoami': IDL.Func([], [IDL.Principal], ['query']),
    });
};

//replace with any canister id
const canisterId = "rrkah-fqaaa-aaaaa-aaaaq-cai";

const agent = new HttpAgent({
    host: "http://localhost:8000",
});
//needed for update calls on local dev env, shouldn't be used in production!
agent.fetchRootKey();

const actor = Actor.createActor(idlFactory, {
    agent,
    canisterId,
});


let callCanister = async (message) => {
    res = await actor.greet(message).catch(e => { return "Error" + e });
    return res;
}

callCanister("TEST").then(res => { console.log(res) });

Running that code gives another error:

TypeError: Cannot read property 'bind' of undefined
    at getDefaultFetch (~/work/dfinity/node_test_2/node_modules/@dfinity/agent/lib/cjs/agent/http/index.js:57:28)

We can again fix this by patching the index.js file, adding the following line after use strict:

global.fetch = require("node-fetch");

And finally, after all this we get the call to work:

~/work/dfinity/node_test_2$ node app.js 
Hello, TEST!

Awesome!

Now, the question is, was I doing something wrong, or are there better ways of calling canisters from node.js? My eventual goal would be to run this on a service like heroku, so it would be great if this can get patched at source. (and of course someone should check if those hacks affect other areas of code)

9 Likes

Excellent writeup! You’re right - node.js workflows are underdocumented, and I’ll add it to my list.

At a cursory reading, you’re not doing anything wrong - we would expect most devs to do the same sort of TextEncoder and fetch setup. I think it can be easier with how to manage the canister ID, but generally you’re doing things right

1 Like

You are a scholar and a gentleman. Thanks for the write-up!

wicked!

Thanks for this … got me over a hurdle … and its working in my local …

I can’t seem to find where the “production” host would point to.
i.e.
if "host: “http://localhost:8000” for the HttpAgent in dev environment, what address do I put in for a canister on the actual IC? …
I looked but could not find easily, and tried a few but cannot get it to work …
Thoughts/input?

Much thanks

I haven’t tried it myself yet, but in the example provided here it’s:

<input type="text" id="hostUrl" value="https://boundary.ic0.app/" />

[...]

const actor = Actor.createActor(idlFactory, {
    agent: new HttpAgent({
      host: hostUrlEl.value,
      identity,
    }),
    canisterId,
  });

Also, I think that not passing any hosturl is defaulting to the public url as well. In the template created with dfx new the declarations .js file seems to work like this:

export const createActor = (canisterId, options) => {
  const agent = new HttpAgent({ ...options?.agentOptions });
 
[...] 
 
  // Creates an actor with using the candid interface and the HttpAgent
  return Actor.createActor(idlFactory, {
    agent,
    canisterId,
    ...options?.actorOptions,
  });
};

 export const test_assets = createActor(canisterId);

I’d try it first without a hosturl, and see if it goes to the public one.

double wicked! it works with the boundary address …

good looking out on the pointer to the example … I should have found that :slight_smile:

I had tried no host which errored … so I chased in the agent lib and the HttpAgent requires it with no reference to what is the production host …

Anyway it works … currently from command line, going to get it into a service now …

THANK YOU,
-M