How do I send a Blob from JS frontend to Motoko backend?

I’m trying to make a request to my Motoko canister. The frontend is written in JS and I’m calling a function within my motoko canister that is expecting a Blob. When Make the call, I’m getting an error message saying that the Blob is formatted wrong. below, is a screenshot of the error message that I’m getting. Any insight would be helpful.

2 Likes

The source is a JSON object? You can convert it as following:

export const toArray = 
       async <T>(data: T): Promise<Array<number>> => {
  const blob: Blob = new Blob([JSON.stringify(data)], 
                         {type: 'application/json; charset=utf-8'});
  return [...new Uint8Array(await blob.arrayBuffer())];
};

(got some other utilities in that blog post TypeScript Utilities For Candid - DEV Community)

2 Likes

I have a full example here including front-end and backend.

8 Likes

Is there anywhere I can go to get detailed documentation on what’s coded in your example? The whole concept of sending files in chunks is still new to me. I’m also coding in Javascript as opposed to typescript, which adds another layer of complexity when reverse engineering the example you sited.

Thanks in advance.

Hi, so in JS you can can convert any file in Blob’s Blob() - Web APIs | MDN and then that blob can be split in smaller chunks due to motoko’s upload size restrictions.
Splitting a File into Chunks with Javascript - Stack Overflow

In canister you can keep everything in 1 canister if you wish so but in my example due to the fact we upload a lot of files so I’ve created a way to keep track of multiple files in multiple canisters.

To keep this simple you can create a hashmap to keep track of the files you upload and then in a different hashmap/triemap you can upload the chunks with the file id . The function that received the chunks should accept the Blob type in motoko. Blob :: Internet Computer

Hope this is helpful.

3 Likes

This does help. I’ll give it another try. Thanks!

what i’ve done here is take the file (as a Blob). from that Blob, I get the arrayBuffer. I immediately use the arrayBuffer to construct a new Unit8Array() (not entirely sure what a unit8Array ism so if you have any insight, please do share :slight_smile: ). I take the unit8Array of the file, and break it into chunks. I construct a Blob out of each chunk, then i send the Blob chunk to the motoko backend where the file upload function in the motoko backend will be expecting an argument of type Blob. How’d I do? am I close? should the file chunks be sent to the motoko backend as a different data type?

1 Like

There might be some quirks in your code to make it work I would say.

One regarding promises handling on line 43 and another regarding the chunking approach itself.

Assuming file in your code is already a Blob, you should chunk this variable first and then do the arrayBuffer conversion on each chunks before uploading to the IC.

For example, I do the following:

const chunkSize = 700000;

  for (let start = 0; start < data.size; start += chunkSize) {
    const chunk: Blob = data.slice(start, start + chunkSize);

    promises.push(
      uploadChunk({
        batchId,
        chunk,
        storageActor
      })
    );
  }

  const chunkIds: {chunkId: bigint}[] = await Promise.all(promises);

In above snippet, data is your file. I construct a list of promises that perform the upload to the IC with chunks of the original blob, the file.

Blob has a buil-tin slice (doc: Blob.slice() - Web APIs | MDN) function.

Then, in the upload function itself, I do the conversion of the chunk to array buffer

const uploadChunk = async ({chunk, batchId}) => { 
  ....

  storageActor.uploadChunk({
    batchId,
    content: [...new Uint8Array(await chunk.arrayBuffer())]
  });

  ...
});

To summarize:

  1. Iterate on the file to create the chunks with slice
  2. Upload each chunk to the IC as array of integer

My upload function starts there :point_right: deckdeckgo/storage.utils.ts at f4699f386d0ab9f474272cfaec3026b20ee604eb · deckgo/deckdeckgo · GitHub

2 Likes

ok. I made the changes to my code to align more similarly with what you have @peterparker.
I get the sense that I’m close, but still not quite there yet. I’m getting a console error. here is my code and below that is the console error that I’m getting. do you see any obvious cause for the console error?

uploadChunk function:

console error:

1 Like

On line 65, can you either try to use await without then or the contrary? does that help?

const result = await Promise.all(promises);
console.log("result from....

or

Promise.all(promises).then((result) => console.log("result...

Another thing, on line 34, isn’t actor.createHournalEntryFile async maybe?

const uploadChunk = async (fileId, chunkId, fileChunk) => {
   await actor.createJournalEntryFile(....

gave it a try. Still getting the errors. Gonna keep at it tho. Thanks for the help thus far! I’m definitely close.

1 Like