ICP LLM Marketplace

Developer Grant Proposals
1

I am applying for a $5k Developer Grant.

Please review my application and discuss! See DFINITY’s directions for becoming a registered reviewer here. They will be collected by DFINITY. When two weeks passes, DFINITY will release them and they will appear as a new section on this post.

Please review my application and discuss! If you would like to submit an official review, then please use the links below to register, see the rubric, and submit a review.

I’m looking forward to everyone’s input!

Reviewer Registration | Rubric for Evaluating | Submit a Review

MY APPLICATION:

REVIEWS:

2 Likes
2

Hi,

I would appreciate some comments on my application.

Best,
Yash.

3

Response to grant review:

  1. Please explain your score for Project Description

The purpose of this product is unclear. Why exacting is a HuggingFace for models that run on ICP is needed and how will users be able to interface with it?

—>It’s a marketplace for models employing a ‘pay as you go’ scheme. A marketplace like this will create an alternate income stream for AIML practitioners and nothing better than deploying it on ICP.

Majority of user input will be abstracted such that they use a simple interface to interact with the model and derive value from it.

  1. Please explain your score for their milestones

They need to commit to certain details first before applying for a grant (For example, they need to decide on the programming language).

—> the entire dapp will run on icp. The frontend will be a nextjs app running as a canister. All the backend modules would also runs a canister written primarily in Rust.

  1. Please explain your score for Feasibility

It is unclear which models they will optimize. If they end up choosing models that we already know can run on ICP, they will not add much value to the ecosystem. If they end up choosing models that might not run on ICP, they risk never finishing the project.

—> this is a great question. Thank you for asking. We are looking at building this as a marketplace that will allow a model creator to upload a model and make it available for consumption. While doing so, earn an incentive.

We plan to explore llama3 (probably the one a colleague in DeAI wg has ported) to start with and build the common infrastructure around it so that only models meeting our stds could be deployed and used. Based on the learning we would cater to building the infra for transformers (these would be comparatively less resource intensive).

  1. Please explain your score for Technical Complexity

Model optimization on an ICP canister can be difficult, in particular regarding performance and cycles burn optimization.

—> ascertaining cycles is a task we have to undertake.

  1. Please explain your score for their reach

They did commit to a HuggingFace-like platform but it is unclear which or how many models they will initially showcase.

—> we plan to build our mvp on llama3. Another transformer model would be our stretch goal.

2 Likes