Local Development with Internet Identity on M1 Apple Processor

Motokoder · February 19, 2022, 8:12am

I have been developing locally using Internet Identity for authentication with no problems on a 2017 iMac with a YubiKey.

I just switched to a MacBook Pro with an M1 processor and set everything up the same, but when I try to login, before I even select the login method, I get the following errors in Chrome Dev Tools (I get a blank screen with Safari):

This happens with the webpack dev server and if I browse directly to the dapp canister: http://{dapp canister id}.localhost:8000.

If I select “This device” and use my fingerprint scanner, it just stays on that screen until it errors with a timeout.

I had no problem compiling and deploying the Internet Identity project. Is there anything special I need to do for an M1 processor to get local authentication working?

Alternatively, is there any code in the Internet Identity project that could be changed to bypass actual authentication and just return a static identity for development purposes?

Thank you.

Detailed error messages:

PoW: 3189.276123046875 ms
index.js:1065 createChallenge(ProofOfWork { timestamp=1645256116345000000, nonce=1420616005839431 })

index.js:2 Invalid asm.js: Unexpected token
await (async)
Dt @ index.js:2
Yt.n @ index.js:2
await in Yt.n (async)
i @ index.js:2
createChallenge @ index.js:1065
await in createChallenge (async)
(anonymous) @ index.js:486
setTimeout (async)
(anonymous) @ index.js:486
t.makeCaptcha @ index.js:486
r.onsubmit @ index.js:1024

index.js:2 Uncaught (in promise) Error: Fail to verify certificate
at Dt (index.js:2:121013)
at async Yt.n (index.js:2:124987)
at async Function.createChallenge (index.js:1065:5993)

Ori · February 19, 2022, 1:58pm

Are you able to login to Internet Identity for apps on mainnet? There were some reports of clock syncing issues on new Macs, a simple reboot might have resolved those though.

Motokoder · February 19, 2022, 2:52pm

Yes, I can login to nns and other dapps. I have not rebooted since posting my question.

Motokoder · February 19, 2022, 4:01pm

@Ori: I just re-deployed my code to mainnet and confirmed that Internet Identity login works as expected from my MacBook Pro (M1 processor).

So this issue is only happening locally with the M1 processor. There is no issue on my iMac (Intel processor). Both macs are running macOS Monterey 12.1.

Here is the mainnet canister:
https://d2dsd-maaaa-aaaag-aabha-cai.ic0.app/login

Here is the readme with the local setup instructions I followed:

github.com

VerifiedGiveaways/vg/blob/develop/README.md

# Verified Giveaways

A decentralized application on the Internet Computer for hosting and participating in fair and transparent giveaways.

- __UI__: React
- __Backend__: Motoko actors/canisters
- __Hosting__: Internet Computer


# Developer Setup

These setup instructions may seem long, but the goal is to make them complete. They should evolve with feedback.

### macOS and Ubuntu Linux

These instructions are for macOS and Ubuntu Linux (not running in a virtual machine).

### Windows

Although Windows can install the Linux subsystem, there are complications with permissions between code editors and browsers running in Windows and the libraries installed via the Linux subsystem. This approach is not recommended.

This file has been truncated. show original

Motokoder · February 19, 2022, 4:28pm

Honestly, I just want a way to mock Internet Identity authentication so that local secure canister calls work exactly the same. If someone has already done this, and is kind enough to share the solution, it would be greatly appreciated!

Motokoder · February 19, 2022, 11:47pm

I have the solution for this and will share it here once I have it refined a bit more.

Motokoder · February 20, 2022, 6:10am

I was able to solve this by copying this file directly into the dApp project, then modifying some functions to return an auto-generated Identity based on a local config setting (the idea came from this post).

github.com

VerifiedGiveaways/vg/blob/1eac799c03bdc4f0c147bdc3fd2be95946e604c0/.env#L10

      
        
            # mainnet environment variables
            # for local development, create a .env.local file with replacement values
            
            
# for local development, use one or both of the following 2 variables
            # II_PROVIDER_USE_FAKE overrides II_PROVIDER_URL
            
            
# local value: "http://<locally deploy Internet Identity canister id>.localhost:8000/#authorize"
            II_PROVIDER_URL="https://identity.ic0.app/#authorize"
            
            
# local value: "true"
            II_PROVIDER_USE_FAKE="false"

github.com

VerifiedGiveaways/vg/blob/1eac799c03bdc4f0c147bdc3fd2be95946e604c0/src/ui/src/components/app.tsx#L27

      
        
            }, []);
            
            
return (           
                <InternetIdentityProvider
                    authClientOptions={{
                        onSuccess: (identity) => console.log(
                            "Successful Login", {identity}
                        ),
                        identityProvider: `${process.env.II_PROVIDER_URL}`,
                    }}
                    fakeProvider={process.env.II_PROVIDER_USE_FAKE == 'true'}
                >
                    <header>
                        <h1>Verified Giveaways</h1>
                    </header>
                    <div className="layout-center">
                        <main className="layout-main">
                            <Outlet/>
                        </main>
                        <nav className="layout-nav">
                            <ul>

github.com

VerifiedGiveaways/vg/blob/1eac799c03bdc4f0c147bdc3fd2be95946e604c0/src/ui/src/context/internet-identity.tsx#L59

      
        
            const [authClient, setAuthClient] = React.useState<AuthClient | null>(null)
            const [isAuthenticated, setIsAuthenticated] = React.useState(false)
            const [error, setError] = React.useState<string | null>(null)
            
            
const identityProvider = (
              authClientOptions.identityProvider || 'https://identity.ic0.app/#authorize'
            ).toString()
            
            
const createAuthClient = React.useCallback(async () => {
              var authClient: React.SetStateAction<AuthClient | null>;
              if (fakeProvider) {
                authClient = await AuthClient.create({ identity: Ed25519KeyIdentity.generate() });
              } else {
                authClient = await AuthClient.create({});
              }
              setAuthClient(authClient)
            }, [])
            
            
React.useEffect(() => {
              createAuthClient()
            }, [createAuthClient])

github.com

VerifiedGiveaways/vg/blob/1eac799c03bdc4f0c147bdc3fd2be95946e604c0/src/ui/src/context/internet-identity.tsx#L102

      
        
              (error) => {
                setError(error)
                onError && onError(error)
              },
              [onError]
            )
            
            
const authenticate = React.useCallback(async () => {
              if (authClient) {
            
            
    if (fakeProvider) {
                  handleOnSuccess(authClient);
                } else {
                  await authClient.login({
                    onSuccess: () => handleOnSuccess(authClient),
                    onError: handleOnError,
                    identityProvider,
                    ...authClientOptions
                  });
                }
              }

github.com

VerifiedGiveaways/vg/blob/1eac799c03bdc4f0c147bdc3fd2be95946e604c0/src/ui/src/context/internet-identity.tsx#L123

      
        
            }, [
              authClient,
              authClientOptions,
              handleOnError,
              handleOnSuccess,
              identityProvider
            ])
            
            
const signout = React.useCallback(async () => {
              if (authClient) {
                if (!fakeProvider) {
                  await authClient.logout();
                }
                setIsAuthenticated(false);
              }
            }, [authClient])
            
            
return {
              error,
              authClient,
              identityProvider,

When developing locally, login just works by using a generated Identity instead of connecting to an II provider.

When deploying to mainnet, login uses the actual mainnet II provider.

Motokoder · February 20, 2022, 7:12am

Uhg. Although this works in the front-end code, when calling a canister function, the caller is the anonymous user principal. . So it seems there are still some things to figure out.

Motokoder · February 20, 2022, 3:59pm

I have been learning about the inner workings of authentication, and my current understanding is that we can’t bypass an II provider because when a user sends a message, it’s signed with the private key of the user’s Identity, then the message envelope includes the signature and the public key.

The IC has to validate the signature against the public key, but it also needs to validate that the principal in the message is the true owner of that public key. I believe that last part means that the IC calls the II provider/canister to match the principal with the public key.

If that is the case, then it seems that a simple mock setup of II authentication for local development would require a mock II canister as well, which just validates all messages/requests.

(You may ask, why not use the II project locally as is, which is my preference, but it does not seem to work on M1 Apple processors. If there is a straightforward solution for the core issue, I would like to use that instead.)

If anyone has read up to this point, thanks for bearing with me.

GLdev · February 20, 2022, 5:53pm

A couple of months ago when I first heard about this issue I googled a bit, and apparently there’s a native VM tool from Canonical that lets you run ubuntu under m1 apple silicon. You could give that a try.

I’ve also had a deep dive through the II authentication flow for porting the agent-js parts to rust with yew, and I believe the idea is that the agent maintains a local random key, but receives a delegation based on that public key, and the replicas just verify that the delegation chain is properly signed. I don’t think the replicas communicate with the II app.

Motokoder · February 20, 2022, 6:09pm

@GLdev Thank you for the response and the tip about Cononical. It’s a good last resort.

I see that delegation is implemented in the II canister. I was just learning about how that works. I think you are correct because the diagrams I have seen show the browser connecting the the II canister, and to the backend canisters, but there does not appear to be communication from the backend canister to the II canister. (Not like OAuth flows.)

It would be very nice to figure out how to mock a delegate. I understand they come with expirations, so that would need to be handled as well.

I really appreciate your help!

Motokoder · February 20, 2022, 8:50pm

Alrighty, I finally have a local Internet Identity canister working on my MacBook Pro with an M1 processor. The trick is to update the JavaScript in the @dfinity/agent library (@kpeacock) from the node_modules folder of your locally cloned Internet Identity project. (It seems that the library is not open-source yet, or I just don’t know how to find it.)

Search node_modules for “Fail to verify certificate”.

Comment out the following code in two places:

/node_modules/@dfinity/agent/lib/cjs/polling/index.js
/node_modules/@dfinity/agent/lib/esm/polling/index.js

async function pollForResponse(agent, canisterId, requestId, strategy) {
    const path = [new TextEncoder().encode('request_status'), requestId];
    const state = await agent.readState(canisterId, { paths: [path] });
    const cert = new certificate_1.Certificate(state, agent);
    const verified = await cert.verify();
    // if (!verified) {
    //     throw new Error('Fail to verify certificate');
    // }
    ...
}

Search node_modules for “UnverifiedCertificateError”.

Comment the following code in two places:

/node_modules/@dfinity/agent/lib/esm/certificate.js
node_modules/@dfinity/agent/lib/cjs/certificate.js

checkState() {
    // if (!this.verified) {
    //     throw new UnverifiedCertificateError();
    // }
}

Delete the dist folder.

Ensure you have a local instance of the IC running:

dfx start --background

Redeploy the II canister:

dfx deploy --no-wallet --argument '(null)'

After that I was able to create a new local Internet Identity. To reiterate from earlier, none of this was necessary on my 2017 iMac with an Intel processor. I don’t know how the processor is related to certificate verification, I just followed the errors and removed the code that threw them.

Motokoder · February 21, 2022, 4:27am

It turns out the fake II provider actually does work. I just needed to pass the identity instance to an actor reference: vg/api.ts at 19245369417f1336c5fcd2c3a0aa9681bb9acad0 · VerifiedGiveaways/vg · GitHub.

With this change, the message caller in public canister functions is the identity that was autogenerated in the UI code.

So now there are two working options: 1. Fake the Identity or 2. Use the local II provider (with the small hack mentioned in my last post for M1/M2 processors).

The code can be greatly improved, but at least this proof-of-concept is working now.

frederikrothenberger · February 21, 2022, 10:32am

Hi @Motokoder

Unfortunately, I’m a little too late to save you the effort, but we recently started providing dev builds of II for local development. This build has the following properties:

Red banner, indicating that it is a dev build
Proof of work disabled for faster browser interaction
Captcha hardcoded to the character ‘a’ so that tests can be automated
WebAuthen replaced by an interaction-less fake implementation
Internet Identity no longer assumes a fixed canister id.

This should allow you to integrate your canister with II the exact same way as it would on mainnet.
Could you quickly check that this works as intended on your M1 mac?

You can grab the built internet_identity_dev.wasm file from one of our CI runs, e.g. here: Replaced internal links to the interface spec by external ones. (#500) · dfinity/internet-identity@594f131 · GitHub

Or build it yourself by setting the following environment variables to 1:

II_FETCH_ROOT_KEY
II_DUMMY_CAPTCHA
II_DUMMY_AUTH
II_DUMMY_POW

And then run the docker-build in the scripts folder.

Best regards
Frederik

Motokoder · February 21, 2022, 1:33pm

Oh nice! Thank you for providing this @frederikrothenberger! I am starting work now, but I will absolutely test this out tonight. .

Motokoder · February 21, 2022, 5:23pm

@frederikrothenberger Can you confirm that this is the correct process for testing the dev build? I am not seeing a red banner and I still get the same certificate error as before. Thanks.

git clone https://github.com/dfinity/internet-identity.git
cd internet-identity
npm ci
dfx start --background

# open new terminal window

dfx deploy --argument '(null)'

# manually copy downloaded dev build of internet_identity.wasm
# into ./.dfx/local/canisters/internet_identity folder.

dfx canister install internet_identity --argument '(null)' --mode reinstall

frederikrothenberger · February 22, 2022, 7:44am

@Motokoder

No, if you do not get a red banner, then dfx canister install did rebuild the II canister.

To run it from the II repository, do the following:

git clone https://github.com/dfinity/internet-identity.git
cd internet-identity
dfx start --background

# open new terminal window
export II_FETCH_ROOT_KEY=1
export II_DUMMY_CAPTCHA=1
export II_DUMMY_AUTH=1
export II_DUMMY_POW=1

npm ci
dfx deploy --argument '(null)'

Also, if you want to run II in your own project (without bothering with the II repo) you can just reference the prebuilt dev internet_identity.wasm in your dfx.json like so:

{
  "canisters": {
    "your_canister": {
      ...
    },
    "internet_identity": {
      "candid": "internet_identity.did", // relative path to II did file. You can grab that from here: https://github.com/dfinity/internet-identity/blob/main/src/internet_identity/internet_identity.did
      "type": "custom",
      "wasm": "internet_identity.wasm" // relative path to dev build wasm
    }
  },
  "networks": {
    "local": {
      "bind": "0.0.0.0:8000",
      "type": "ephemeral"
    }
  },
  "version": 1
}

I hope this helps.

Motokoder · February 22, 2022, 4:35pm

@frederikrothenberger Thank you for the step-by-step instructions. I have tested on my MacBook Pro 2021 with M1 processor and recorded the results. It works nicely for creating a single identity, however, there is an error when creating a second. I’m not sure if it has anything to do with my code, so I have included that in the video as well.
https://dwqte-viaaa-aaaai-qaufq-cai.ic0.app/1458-944/internet-computer-development-with-a-local-internet-identity-canister

Unfortunately, DSocial is not loading the video, so here’s a YouTube upload (HD still processing): Internet Computer Development with a Local Internet Identity Canister - YouTube

kpeacock · February 22, 2022, 6:35pm

There’s also going to be a new "remote" config coming to dfx.json in the next version. Together with the development build, we should be able to simplify the developer experience for identity in the near future

Motokoder · February 22, 2022, 10:19pm

I look forward to it Kyle! Thank you!

Topic		Replies	Views
Setup Internet identity locally failed - MacBook M1 Developers Discussing , community-consideration	4	1838	November 16, 2021
Trying the build the Internet Identity canister locally on MacOS. Is it possible? Language Support	7	1547	December 7, 2021
Local Internet-identity front-end build is failing Developers	27	2308	November 28, 2022
Issues during local deploy of Internet Identity Developers	8	1099	February 9, 2023
Local version of the internet-identity isn't responding when i try to create a new internet identity anchor Developers	9	1196	April 1, 2022

Local Development with Internet Identity on M1 Apple Processor

Search node_modules for “Fail to verify certificate”.

Search node_modules for “UnverifiedCertificateError”.

Delete the dist folder.

Ensure you have a local instance of the IC running:

Redeploy the II canister:

Related topics