Motoko Binary Concatenation

Is there a reason that # or + doesn’t work for binary concatenation? I had an issue over the weekend where I was trying to push content in via the dfx command line. You are limited to about 1MB on the command line. A 400KB image encoded as “\FF\FF…” over runs that. I tried to use Array.append but I eventually hit the cycle limit when my hash map got over a certain point.

It would be great to have a memory efficient append for binary data. It will also be important if you are trying to build up a large file of a certain format that needs to be encoded.

The whole story around Blob is a bit unsatisfying, but discussion on where it should go (should it be more like arrays maybe) has a bit stalled: Quo vadis `blob` · Issue #966 · dfinity/motoko · GitHub

That said, maybe you don’t actually want to concatenate binary data (which requires a bunch of boring byte copying), but merely aggregate it without copying (e.g. a list of blobs), so that you can stream it when needed? Depends on the concrete use case, of course.

2 Likes