check out this interesting :
https://internetcomputer.org/blog/news-and-updates/team-spotlight-node
@lastmjs ,
I am digging into the use of wasi2ic within both Kybra and Azle.
I noticed that both are using the feature transient
for the ic-wasi-polyfill, with reference to this issue:
What exactly is the impact of using this feature, and do you know if there is work on-going to fix it?
If you turn that off the polyfill will use the stable memory file system. We tried it and unfortunately found it to be unstable (ha) in various ways that did not make us feel comfortable using it yet.
@sgaflv can give you more details on timelines and current state of the polyfill and its stability.
@lastmjs I am currently working on improving the work of stable-fs. I will provide a new version by the end of this month and will check that the issues are resolved.
Just a quick note that I was able to integrate the polyfill and wasi2ic into icpp-pro, the CDK for C++.
It is all still in my dev environment, and I need to do a lot more testing, but first experiments show that it is working really well also in the C++ world, and I believe this is going to help a lot with porting C++ libraries to the IC.
I took a lot of inspiration from kybra, and icpp-pro now installs rust + rust dependencies in a .icpp
subfolder of the userās home directory.
When a user builds their wasm, everything is transparently included. They donāt need to do anything to their code.
Some things that work now (tested it):
@sgaflv ,
Will stable-fs also work for a C++ canister?
If so, do you happen to have a simple demo example available? *(please?)*
The demo2 that you created was extremely helpful in figuring out how to get it all to work.
yes, it should work with the file system as well. Sounds like a good idea to create an extra demo for that (or maybe make the demo2 more advanced). I will add an issue for that.
I am trying out an idea where the user keeps a local copy of sqlite in sync with the main copy running inside the canister. It might be an interesting design pattern for personal apps where the db is known to be small.
All works well, mostly. I am not able to upgrade the canister. The DB is still there after the upgrade but seems to have lost one of its tables.
#[init]
fn init() {
ic_wasi_polyfill::init(&[0u8; 32], &[]);
DB.with_borrow_mut(|db| {
*db = Some(Connection::open(DB_FILE).unwrap());
let db = db.as_mut().unwrap();
modules::change::init_db(db);
modules::chat::init_db(db);
});
}
#[post_upgrade]
fn post_upgrade() {
ic_wasi_polyfill::init(&[0u8; 32], &[]);
DB.with_borrow_mut(|db| {
*db = Some(Connection::open(DB_FILE).unwrap());
});
}
Am I missing something here? Or maybe issues are related to the unstableness described by @lastmjs above?
Frontend: https://vicki-naaaa-aaaal-qjgma-cai.icp0.io
Backend: DFINITY Canister Candid UI
@kristofer ,
I wonder if your issue is related to something @ulan fixed and described here
Thanks for the tip. Yes, assigning the wasi memory a MemoryId
sounds like good practice. I added that, problem remains though.
More details on the sqlite error code below. Seems sqlite canāt access the db file after upgrade. I have verified using std:fs
that the file still exists and file size remains unchanged etc.
(1802) SQLITE_IOERR_FSTAT
The SQLITE_IOERR_FSTAT error code is an extended error code for
SQLITE_IOERR indicating an I/O error in the VFS layer while trying to
invoke fstat() (or the equivalent) on a file in order to determine
information such as the file size or access permissions.
Hello Kristofer,
Iāve checked your example. Unfortunately there seems for be a bug in the way the stable-fs is storing the data in stable structures, which leads to loosing files after canister upgrade. I am working on a fix now.
You have an awesome sqlite example! If you donāt mind, Iād mention it on the wasi2ic page.
Thanks for looking into this! And yes, please do mention my example if you will.
Perhaps canister upgrade could be added to your demo3 as well?
@kristofer
Good point, Iāll update demo3. Check out the latest published ic-wasi-polyfill version 0.4.3, see if it resolves your issue.
Yes, great! Issue resolved!
@lastmjs: fyi, this should also fix the flaky issue you were observing. ^
@sgaflv ,
FYI, the integration of ic-wasi-polyfill & wasi2ic into icpp-pro is finished, and released (as a pre-release initially, but it is working great so far)
You can check out the wonderfully simple C++ code, with stdio, file-io, file upload/download that this now enables in the icpp-demos/canisters/files example canister.
Also fyi, with extreme little effort I was now able to get files to upload to a C++ LLM that I am working on.
I canāt thank you enough for building wasi2ic.
Yes itās amazing! I really hope it can all get integrated into dfx eventually so that Wasi becomes the defacto standard on ICP and it just works.
@sgaflv ,
from C++, we call raw_init, like this:
int main() {
raw_init(nullptr, 0);
return 0;
}
Is that still correct?
I am concerned about the discussion above regarding Rust based aplications, that one should call init_with_memory or init_with_memory_manager.
Not sure if this applies to C++ as well.
@icpp it is only important if your C++ application also wants to directly access the stable memory. The Rust memory manager allows to share the stable memory between the application and the ic-wasi-polyfill library such that they donāt overwrite each otherās data in the stable memory.
If your C++ application stores data in the stable memory using the standard file system and doesnāt call the stable memory API functions, then it is all good and will work correctly because ic-wasi-polyfill will manage the stable memory.
Iām really fascinated by the technology behind this!. Could you explain how WASI fits into the ICP ecosystem? Iām curious about what this enables and how it is better than other WASI usages.