the two repos I have tried last year, can not work on IC, because the low level of them are written in C++ and can not be compiled into wasm format.
So now, I’m coding my own neural network library based on Rust in my spare time and it can be boxed into the canister.
3 Likes
clang can target wasm, so it shouldn’t be impossible to port the C++ libraries. But it may be more effort than it is worth. If you’re interested, have a look at a c canister example here (not in working state at the moment IIRC)
1 Like
thx, I will try it later.
BTW.
Though it can be compiled into wasm format, how can we generate the candid file?
By writing a generator yourself I suppose… Or maybe one of the C++ CDKs offers one already. There’s a few ones you can discover through forum search.
For the beginning it’s probably easiest to write it by hand. That’s how I started in my first Rust canistrers too
3 Likes
what about burn burn/examples/mnist-inference-web at main · burn-rs/burn · GitHub ?
3 Likes
Hi mnl, I have researched the burn last week, it is also based on tch-rs which low-level is written in C++.
also based on tch-rs
I don’t believe that’s the case, tch-rs is only one of the backends offered by the burn crate see features:
The NdArray backend is used for compiling into WASM and it’s the thing used in that MNIST inference web example I linked in my previous post
1 Like
There is also dfdx which compiles to no_std therefore it shouldn’t be a problem to compile it to wasm. I believe It’s being used in e.g. GitHub - Narsil/fast_gpt2 to provide inference
1 Like
Ok, so if it’s possible to compile it into a wasm file, then the next step is to research how to generate candid file.
2 Likes
Found a repository of ML Libraries for Rust and some bindings to popular Python frameworks. Huggingface tokenizer and SentenceTransformer would be really cool in the IC!
1 Like
Hello everyone,
I recently had an idea about incentivizing miners to help machine learning engineers and AI researchers train deep learning models. Instead of using computation power to guess numbers for SHA256, the idea is to switch to computing the loss function threshold, which is a measure of how well an AI model performs on given inputs.
Here’s how it would work:
- A machine learning engineer submits a deep learning model implementation to the blockchain and requests for training. The value of the loss function of the AI is easy to compute given some input.
- The node providers see the assignment and begin training the models.
- Suppose a node provider has superior hardware and is able to train the AI faster than other miners. Once they finish training, they start broadcasting the weights of the model to other miners, which is the result of a well-trained AI.
- Other miners receive the weights and compute the loss function and the Delta of the loss function. If both values are below some threshold that the machine learning engineer required, then the AI is considered good enough and can no longer be significantly optimized.
- If the other node providers agree that the AI is well-trained, then consensus is reached, and the node provider who trained the AI receives tokens as an incentive.
The main idea is to switch from the number guessing game to computing the loss value, while keeping the rest of the process the same.
However, there are some unclear parts, such as whether validators who verify the value of the loss function should be incentivized. According to the logic of BTC, they should not. Also, not every machine learning engineer may be able to define a robust AI, and it is possible that the loss can never reach the required threshold. For example, the gradient may not be able to go down because the data or model is not good enough.
What do you think about this idea?
6 Likes
I have a list of questions:
1.
What do you mean by “submits a deep learning model implementation”? Is that mean the train source code? And if that can implement on IC, the format do you want to store the code on IC? How do you input the model code?
1 Like
I did actually try to train a NN onchain a few months ago here → GitHub - SwayStar123/nn
it worked well and fine, except it was a tiny neural network for recognizing hand written digits (trained on the mnist dataset). Since it was a small neural network, training it on the cpu wasnt a big deal, but itd be impossible to train stable diffusion type models on chain, you need GPUs for that, and yeah since you would need to come to consensus for it, seems like a waste of resources.
Querying for inference seems like an excellent usecase however, since only one node needs to do the calculations for that, and the required hardware is much lower during inference time
6 Likes
Yes, it may be necessary to provide the training source code along with the training data, and on IC it might need to be implemented using Rust or Motoko. Solidity would not work because it does not even have a float data type.
1 Like
Great job!
In your code, it seems like for every training step we need to perform consensus. And you store the weights in canisters, right?
Yes to both the questions
2 Likes
I really like the idea of taking market-making to the next level, using AI on the blockchain. There is still a question of which liquidity pools to add liquidity to, how to add more positions when the liquidity price changes, e.g. for Uniswap V3. Imagine having an AI agent, perhaps using outcalls to query for results, that would then be able to add liquidity in a dynamic fashion for users. Then connect this to the #Bitfinity EVM.
1 Like
Nice effort! I tried to make it possible to execute canister’s train and predict using PyScript.
It seems that there are still many challenges, but I am looking forward to the future.
nn with pyscript
4 Likes
I’ve spent a moment gluing the default dfx project together with the example provided by the burn project: burn/examples/mnist-inference-web at main · burn-rs/burn · GitHub :
8 Likes