Knowing that AI is not perfect but is always improving, would you pay between $10-$100 to upload your ICP-specific codebase to an AI tool that will generate an ICP-aware security audit? Basically your code goes in → markdown audit report comes out.
Yes
No
0voters
Background Information
Hi everyone,
As we’re nearing the release of Azle 1.0, our focus has shifted to internal security reviews. This takes a lot of manual human labor. We essentially are looking over every relevant file and checking it manually against the appropriate Common Weakness Enumerations (CWEs). I will also be doing appropriate passes to check for ICP-specific vulnerabilities.
I’ve been contemplating doing a fully-automated AI pass of the codebase as well. This would involve building a pipeline using one of the major model provider’s APIs. It would accept a codebase, walk through each relevant file, and check against the CWEs. It would be provided with the context of the entire codebase, information for each CWE, and ICP-specific vulnerabilities. Polished output would be a markdown file or set of markdown files with the security audit report.
If I’m going to build this tool for Azle’s internal purposes, I’m wondering if anyone would pay for it if it were hosted. I’m thinking we would probably open-source the tool no matter what, but would charge for a nice hosted web interface or CI/CD API tool.
I think it depends how good it is, but in principle this could be very useful. Particularly if it can help avoid some of the bugs and hacks that have occured in ecosystem projects of late.
Longer term, if this sort of service were ever hosted onchain, it could certify canisters, giving an extra level of confidence to users.
I’m not sure, @marc0olo may be the best person to estimate.
I personally think the IC really, really needs some sort of minimum standards certification system. People using dapps on the IC should have an easy way of knowinging if the service is fully IC-hosted, or if it will fall over when something like cloudflare runs into issues (for example).
hard to give concrete numbers. but that said, every project I talked to that deals with privacy focused features or secures funds of users and wants to seriously ship their canisters into production is interested in getting their code audited.
security audits in general are quite expensive, so I think depending on how good the AI Audit Tool is, most of those projects would be happy to get a report for 10-100 USD.
I personally would use it if I see value in it. I’d probably want to see some examples of the tool first to validate that it provides value.
I think this is a completely different topic. there are also projects where only small parts of the whole application needs to run fully on-chain. I think such audit tool should focus on canister code only.
I personally don’t think so. It’s all under the umbrella of better labelling and warnings to users about what they can expect from a particular dapp. It’s then up to the user what they do with that information. Note that a malicious js file served from an off-chain source could completely transform the UI and/or control a users assets attached to that domain, if a developer (or someone else) has control of that off-chain source. Similarly, DNS could be changed to point to a different site.
Off-chain models will always be more capable than any on-chain model. I don’t think its about what the developer is directly getting out of something like this - it’s about what the user is getting (as far as i understand).