Hello everybody.
I would like to introduce a simple solution that allows you to store any kind of files, organize them, manage them etc. It is a first version which is made up only from the backend canisters. But frontend canister and more opportunities could be added later and other services could be built on top of this service. It is a beginning.
Preface
Very often I use AWS S3 storage to simply store different files for my working needs. Sometimes I put a simple html file(s) here and easily share this html for some internal demo etc. Yes, there are other cloud platforms where the same “magic is possible”.
When I started to review capabilities of the ICP platform I understood that storage services like AWS S3 or Google Drive could be implemented in the ICP platform. Since ICP allows not only to store the data but also executes various computations as well, and also supports cool features like tESDSA and VETKD then several interesting decentralized applications should appear in our lives soon. I hope so.
But right now, I want to show my first journey with ICP platform
Idea and key terms
Right now the project consists of backend canisters only but a simple HTML interface is also supported (thanks to capabilities of ICP canisters to handle http request).
The first name is a storage service, and the main canister has the same name. But I want to apply some renaming later and use the term ics2 instead.
StorageService → main canister that is responsible to manage the customers who can utilize the service and how. System supports the idea of Tiers to control the capabilities of the customers. Customer is an entity who can work with the service. Customer has a tier. Tier tells what is allowed, what is not (nested directories, number of applications, private repositories etc). If you want to apply extra restrictions for the customers or extra management or “simple monitization” of the access then it is a right place to do such changes here.
Application → think about application as a virtual machine or isolated partition inside something.
Customers may have more than one application. Customer can create an application when it is needed (sure, if the assigned Tier allows to create a one more)
Each application is a backend canister and it exposes a simple html interface (thanks to capabilities of ICP canisters to handle http request).
The idea of this html interface is an easy way to inspect what do we have inside an application
Here is an [exampe] (https://u2s5k-liaaa-aaaak-affoa-cai.raw.icp0.io/i/) of the application html interface.
It is possible to navigate over any repository, go to the data bucket, open any directory, navigate over the subdirectories etc.
Repository → it is not a canister, it is a logical unit inside an application canister. The analogy is a “drive a, b, c, d” in my laptop. Number of repositories is limited by the assigned Trier level.
Repositories could be public and private.
Application canister supports a direct link to the repository. You can share this link, repository page displays all databuckets (if more that 1 used)
Public repositories – application owner can write data, and anyone can read the data (http interface), html interface renders the entire structure of the repository.
Private repositories – application owner can write data. The read operation is allowed by http interface ONLY if the request contains a valid apiKey. Application owner can manage the apiKeys and distribute them if needed. Default html interface doesn’t render the directories/files if the http request doesn’t contain a valid apiKey.
DataBucket → it is a backend canister that is responsible for storing all files and organizing the directories and subdirectories. Each file always has an internal key and the file is always accessible by its key (raw link). If files are stored without any directory then duplication of the file names are allowed (for now). But if files are stored in the directory, then duplication of the names is not allowed inside that directory. It was designed to allow “access to the resource” by its name if files are under the directory. In any way, the raw link is always available. But the access by the “names” give more extra value.
Here is a link to the directory with some books:
https://u5t36-gqaaa-aaaak-affoq-cai.raw.icp0.io/i/books
You can also naviage over the subdirectories
Here is a link to the directory with some images:
https://u5t36-gqaaa-aaaak-affoq-cai.raw.icp0.io/i/Images
Here is a link to the subdirectory of Neuchatel town (Switzerland)
https://uuqqc-qyaaa-aaaak-affpa-cai.raw.icp0.io/i/Switzerland/Neuchatel
Custom template
There is an opportunity to apply some custom template around the html files that are uploaded to the application (repository).
For example, you can inject custom ad banner or custom image etc.
For my demo needs I create a lot of simple articles.
Here is an example of the article with a default template
But in a one moment I can apply “widgets” to sell NFTs
Or inject something like that
It worths to notice, that custom template is applied for the html files and is propagated only to the repository. So, you can apply a template per a repository.
In fact, template doesn’t change the original content of the uploaded html file. But it just “wrap” response during rendering html file by the backend canister
Features
-
Service allows you to organize and store your public content, instantly host simple articles, make your documents accessible for everyone, store a copy of your music etc.
-
You can download or open the file by http interface
-
You can create a private repository as well and restrict the access by apiKey.
-
All uploaded content has a permanent link to download the file or open in the browser.
-
Application owner can organize the content between logical repositories and different canisters (databuckets). Each repository has its own “scaling strategy” (when a new databucket should be created).
-
Each file or directory may have TTL attribute, so resources can be removed automatically.
-
Right now, the frontend canister is not implemented but the minimum html interface is served by the backend canister. This interface allows you to inspect the content, open directories, navigate over them etc.
-
Also, if you store an html file (like an article) then you can apply a “custom” template around it. For example, you can put any ad banner in the left or right side or any other material. It depends on your imagination.
Think about the ics2 project as an abstraction to store and organize files. Other projects could be built on top of it. Sure, I have some ideas about what should be extended as well.
Possible new features
-
Frontend canister to give better UI interface
-
Opportunity to compress files of the directory into zip file and download the entire directory as a zip file. Good feature , but didn’t review the existing libraries in Motoko. If anyone knows how to compress files into a zip file for further downloading, please let me know.
-
Opportunity to replace a content of the existing file (till 2 mb). I’ve already put some changes into the trunk version
-
Opportunity to lock a directory/file from any modification (for some period) including a nested directories as well. I’ve already put some changes into the trunk version
-
Opportunity to share “write” access between different participants for some directories.
-
Authentication with II (and by a pem file) and support of Ethereum login. It would be cool if the user can sign in with his ethereum address and create an application for his needs. Yes, frontend canister and backend canister changes needed, but the strategy is clear. Shared access for the directories, connect evm-based users as well → it should be pretty interesting.
-
Capabilities of the default html interface served by backend canister could be extedned as well. I suppose, that repository unit can optionally have a logo, extra counters could be useful as well.
-
If project supports “shared directories for write access” then it also will be reflected on the default html interface.
-
If “shared directory” is supported and “login with ethereum and II is supported” as well then I would add something like “my contacts” and related flows
-
And more…
Features to highlight
I want to draw a bit more attention on some features and technologies that could be useful for the project.
-
Shared write access. As an application owner I want to give create a directory and allow to other participants to store some files here. It is very common case.
-
Encrypt content with VetKeys technology. As an application owner I want to store some files as encrypted till some criteria or due any reason. I dont wan’t to create a private repo where an access is controlled by apiKey. No, I want to use public repo, it is ok for me. But I want to encrypt some text files.
For that case the VetKeys capability should work well. Yes, UI is needed for the end to end flow. -
Sign in with ICP world (II) and Ethereum world. Apparently after some time UI canister will appear. So, it is useful to allow a sign in with ICP and Ethereum (ethereum login). Yes, backend canister should be extended to support “ethereum authorization, session key”, but it is not a problem.
So, participant is a person or service authenticated by ICP or EVM-based platforms.
Participant can create an appliccation, create a repository and even include “contributer” for any folder.
Examples
Here is a default html interface of the application canister. It has two repositories (for now)
Showcase_workspace – repository (aka logical drive) to store different files : video clips, pdfs, images, etc. Maybe later I will upload more files. All files were uploaded just for the demo needs
Travel_content – repository to store some articles, travel content. All materials were created for the demo needs.
You can observe clickable elements in this UI interface. If you click upon the bucket link, then you are being redirected to the structure of the bucket and see the files and directories.
Just a list of links
Here is a link to open a pdf file
Here is a link to download a file
https://u5t36-gqaaa-aaaak-affoq-cai.raw.icp0.io/d/16d0c2be746a5b58255ce1bc2030b3381c6151362638bff225c113b129a39413
Here is link to the directory (with subdirectories) with images
https://u5t36-gqaaa-aaaak-affoq-cai.raw.icp0.io/i/Images
Link to open an image
Here is a link to the directory with some places of city Neuchatel (travel articles)
https://uuqqc-qyaaa-aaaak-affpa-cai.raw.icp0.io/i/Switzerland/Neuchatel
Here is a link to the simple article that fully stored on chain
https://uuqqc-qyaaa-aaaak-affpa-cai.raw.icp0.io/i/Switzerland/Neuchatel/Red%20Church/index.html
Here is a link to the video clip
