Generated declarations in Node.js environment break

Developers
1

Here is some of the JS generated with dfx generate that I have and want to use in a Node.js environment:

import { Actor, HttpAgent } from "@dfinity/agent";

// Imports and re-exports candid interface
import { idlFactory } from './management_canister.did.js';
export { idlFactory } from './management_canister.did.js';
// CANISTER_ID is replaced by webpack based on node environment
export const canisterId = process.env.MANAGEMENT_CANISTER_CANISTER_ID;

/**
 * 
 * @param {string | import("@dfinity/principal").Principal} canisterId Canister ID of Agent
 * @param {{agentOptions?: import("@dfinity/agent").HttpAgentOptions; actorOptions?: import("@dfinity/agent").ActorConfig}} [options]
 * @return {import("@dfinity/agent").ActorSubclass<import("./management_canister.did.js")._SERVICE>}
 */
 export const createActor = (canisterId, options) => {
  const agent = new HttpAgent({
    ...options?.agentOptions,
    // host: 'http://localhost:8000'
  });
  
  // Fetch root key for certificate validation during development
  if(process.env.NODE_ENV !== "production") {
    agent.fetchRootKey().catch(err=>{
      console.warn("Unable to fetch root key. Check to ensure that your local replica is running");
      console.error(err);
    });
  }

  // Creates an actor with using the candid interface and the HttpAgent
  return Actor.createActor(idlFactory, {
    agent,
    canisterId,
    ...options?.actorOptions,
  });
};
  
/**
 * A ready-to-use agent for the management_canister canister
 * @type {import("@dfinity/agent").ActorSubclass<import("./management_canister.did.js")._SERVICE>}
 */
 export const management_canister = createActor(canisterId);

The problem here is that createActor is called when importing the file, but createActor needs some more properties (like host) to run correctly in a Node.js environment. This renders it impossible to use this automatically generated code without modifying it manually.

Here’s what I have to do to use createActor correctly in Node.js:

const management_canister = createActor(
    'rrkah-fqaaa-aaaaa-aaaaq-cai', {
        agentOptions: {
            host: 'http://localhost:8000'
        }
    }
);

Is there a way around this? Otherwise I think this is a bug.

2 Likes
2

@kpeacock does this look like an issue to you?

3

Can’t you call Actor.createActor manually?

The exported createActor function looks like it’s only providing some convenience.

4

It is an issue - I have a work item to create a node-friendly export option with a configuration setting in the dfx.json config, or simply to disable the default export.

As @paulyoung said though, the index file is mainly for convenience and education - it’s not possible to solve all use cases with code generation. I recommend writing your own createActor constructor in a separate file and importing the .did.js declarations, as I do in this example codebase:

4 Likes
5

Thanks! Though the Node.js environment breaking for IMO simple reasons is something that I imagine can be fixed without too much effort.

The global fetch issue and being forced to pass in a host I believe are the two main differences between Node.js and the browser.

2 Likes
6

Agreed. Also, global fetch is fixed in Node 18, which I can now recommend after testing it last week

1 Like
7

I’ll try Node 18 in just a couple minutes, that would be great news

8

I just switched to Node 18 and removed the following code in my tests:

import fetch from 'node-fetch';
(global as any).fetch = fetch;

All I did was remove those two lines and switch to Node 18. Now I get the following error:

Unable to fetch root key. Check to ensure that your local replica is running
TypeError: fetch failed
    at Object.processResponse (node:internal/deps/undici/undici:5575:34)
    at node:internal/deps/undici/undici:5901:42
    at node:internal/process/task_queues:140:7
    at AsyncResource.runInAsyncScope (node:async_hooks:202:9)
    at AsyncResource.runMicrotask (node:internal/process/task_queues:137:8)
    at processTicksAndRejections (node:internal/process/task_queues:95:5) {
  cause: ConnectTimeoutError: Connect Timeout Error
      at Timeout.onConnectTimeout [as _onTimeout] (node:internal/deps/undici/undici:2282:28)
      at listOnTimeout (node:internal/timers:566:11)
      at processTimers (node:internal/timers:507:7) {
    code: 'UND_ERR_CONNECT_TIMEOUT'
  }
}
 test update failed Error: Fail to verify certificateompleted in 2ms

In addition to that Node 18 gives this warning:

(node:246264) ExperimentalWarning: The Fetch API is an experimental feature. This feature could change at any time

Maybe it isn’t best to rely on that yet?

9

You also need to use 127.0.0.1 instead of localhost, I’ve discovered

10

That did not work unfortunately, here’s my setup:

import {
    run_tests,
    Test
} from 'azle/test/new-test';
import { createActor } from '../src/dfx_generated/update';

const update_canister = createActor(
    'rrkah-fqaaa-aaaaa-aaaaq-cai', {
        agentOptions: {
            host: 'http://127.0.0.1:8000'
        }
    }
);
11

Odd - that looks fine to me

12

Actually, it looks like the problem is because in my tests I do a dfx deploy and then immediately use @dfinity/agent to do some calls. In Node 14 everything worked, but in Node 18 there is a timeout…I’m debugging to find out what’s going on. If I don’t deploy first I can call the canister methods just fine.

13

For your debugging purposes - the fetchRootKey failure tends to be because the local replica http proxy itself is not responsive for whatever reason - it’s prior to any specific canister interactions

14

Ah, it’s happening just when I do a dfx deploy for some reason…

15

I think I know what’s wrong, let me post here to show what’s up

16

Here’s the full code for the tests for a simple canister:

import { execSync } from 'child_process';
import {
    run_tests,
    Test
} from 'azle/test/new-test';
import { createActor } from '../src/dfx_generated/update';

const update_canister = createActor(
    'rrkah-fqaaa-aaaaa-aaaaq-cai', {
        agentOptions: {
            host: 'http://127.0.0.1:8000'
        }
    }
);

const tests: Test[] = [
    {
        name: 'deploy',
        prep: async () => {
            execSync(`dfx deploy`, {
                stdio: 'inherit'
            });
        }
    },
    {
        name: 'update',
        test: async () => {
            const result = await update_canister.update('Why hello there');

            return {
                ok: result === undefined
            };
        }
    },
    {
        name: 'query',
        test: async () => {
            const result = await update_canister.query();

            return {
                ok: result === 'Why hello there'
            };
        }
    }
];

run_tests(tests);

run_tests will execute each test in order, and it will wait for one test to complete before going on to the next test. Here’s what I think the problem is. createActor does an async request to the replica, but when calling createActor you can’t await it at the point of calling it. So, my first test was calling dfx deploy just after my code called createActor, and thus createActor could not call fetchRootKey.

I would suggest fixing this somehow…either forcing the user to await createActor or something, but that was confusing.

And perhaps this manifested in Node 18 because it’s just faster for some reason.

17

This is going to add a few seconds (5 for now just to be sure) on all of my tests because I don’t have a good way of knowing when createActor has finished everything that it needs to. If it returned a promise then I could wait properly.

18

This whole fetchRootKey thing seems to cause a lot of problems, it would be nice to just get rid of that concept locally somehow. I would imagine there’s a more elegant solution available than dealing with it like we are currently.

19

btw localhost is working fine for me in Node 18:

const update_canister = createActor(
    'rrkah-fqaaa-aaaaa-aaaaq-cai', {
        agentOptions: {
            host: 'http://localhost:8000'
        }
    }
);
20

For that, we’d need to know the context that we’re operating in - I don’t want to bake in process.env assumptions into the actual agent-js itself.

Furthermore, there’s no way to know what the root key of a local environment is without making an async call to the network, since it will change each time it is spun up. The only network we can be sure of is mainnet. I feel the pain, but I don’t know of a way around it.

If it returned a promise then I could wait properly.

That should be doable - it would break the existing api if we changed it in the declarations, but you can always define your own actor with

export const createActor = async (canisterId, options) => {
  const agent = new HttpAgent({ ...options?.agentOptions });

  // Fetch root key for certificate validation during development
  if (process.env.NODE_ENV !== "production") {
    await agent.fetchRootKey().catch((err) => {
      console.warn(
        "Unable to fetch root key. Check to ensure that your local replica is running"
      );
      console.error(err);
    });
  }

  // Creates an actor with using the candid interface and the HttpAgent
  return Actor.createActor(idlFactory, {
    agent,
    canisterId,
    ...options?.actorOptions,
  });
};
2 Likes