Howdy Y’all! I wanted to give y’all a friendly intro to the Constellation Book “development dapp”.
We went ahead and built ourselves a socialfi dapp, like Twitter but with modular data sovereignty based on the Actor model. She’s got tutorials and everything to help get y’all up to speed.
Now here’s some more details about this here dapp in case you’re interested:
We’re cultivating a community of dapp developers learnin’ and buildin’ together. The dapp lets you share posts, follow friends, like and comment on their posts too. Real social-like.
But we got data ownership - your content is yours. We divide it up logical-like into “canisters” so you control your piece. And it’s all decentralized on the IC blockchain. No big bosses.
We’re currently developing the base code and tutorials, aiming to launch mid-January 2024, so stay tuned!
We’re hoping to grow a community of dapp devs like you who want to learn and create the future of the web. So mosey on in and join us!
Let me know if y’all have any other questions! Happy to jaw with ya.
👇The following reveals details about this dapp. 👇
Proton - A SocialFi Dapp
Slow is fast.
Decentralization.
Now we are designing a decentralized social media called Proton based on the Actor model. First we need to understand the basic principles of the Actor model. The Actor model is a concurrent computing model that achieves parallel and distributed computing through message passing and asynchronous processing.
So when designing a Dapp, each Canister should be responsible for a different functional module. For example, some Canisters are responsible for recording user profiles, some for storing posts. Also, pay attention to the scalability of the Canisters. We can dynamically create Canisters of the same type to handle high loads.
Design Philosophy
We want to build a truly open Web3 dapp that is modular and data sovereign based on the Actor model.
We hope that each user has their own independent space (Feed Canister) that they fully control. Users can even directly deploy their own independent Feed Canister with code to interact with Proton. (This is cumbersome, only suitable for programmer users, who can develop advanced custom features for the container.) This allows the community to create custom advanced features.
Idea
First there is a scalable public area that receives posts from all users. There is also a user area that records user registration, personal profiles, and following relationships.
We create a Feed for each user to store their own information flow. The Feed is also the user’s private space. Users can store posts in their own Canister (Feed). Except for themselves, no one else can delete it.
The interaction between users and the public area is automatically handled by the Feed Canister. Users only need to query their own Feed to access the latest information stream they are following. Posting, commenting, and liking interactions are also automatically completed by Feed after the initial action.
Users can also add some advanced custom features by deploying their own independent Feed to interact with the public area. For example, only send posts point-to-point to some Feeds, establishing private small social circles; or only connect AI for automatic posting, etc. Any function can be implemented. The community can develop secondary and freely expand various functions. For example, adding a point-to-point private messaging feature.
This is a completely open, slightly slower decentralized application. This design sacrifices some speed for decentralization, just like Bitcoin.
The advantage is that the user’s front end only needs to query their own Feed to get posts from people they follow. It’s convenient and fast. Everything in the background is completed by decentralized collaboration between Canisters, completely decoupled. If a few Canisters go down, it does not affect the continued operation of this system. (If Fetch goes down, you can create a few more)
If the system cannot be recovered temporarily, the Feed can send posts in batches directly to followers using ignore call. That is, two post delivery processes are built into the Feed: posting through the Fetch relay station, and point-to-point posting.
Message Transmission Process
When a user posts, the Feed first stores the post in its own information flow, and then sends the post point-to-point to the follower’s Feeds and the public area according to the followers list. But what if there are 10,000 followers? The situation is not so good, because message sending and receiving between Canisters is limited by max input/output queue size, and cannot send so many at once. It would take the Feed a long time to send them all in batches.
To increase throughput, we add a message relay station: Fetch. The Feed first sends the post to the public area, then sends the post ID and followers to Fetch. Fetch records it, and then notifies these followers’ Feeds according to the algorithm to fetch which posts, and finally the Feed fetches the posts from the public area.
This way, even with many followers, they can fetch posts from the public area in turn under the coordination of Fetch.
Automatic Pressure Adjustment
After Fetch accumulates a certain number of messages, it adjusts the notification order and interval through algorithms (which Feed to notify first and how many milliseconds to wait before notifying other Feeds after each notification), to ensure the query pressure on the public area cannot be too high. When the Feed receives the notification, if the fetch fails, it should wait 20 seconds before trying again.
If the public area faces too much query pressure, it will tell the Root Fetch: “Slow down notifying Feeds”, and the Root Fetch will notify the Fetches under it to reduce notification frequency. If the pressure remains high after 10 minutes, the Root Post can also create a new Bucket to send new posts to.
Also, if the number of users increases, Fetch itself can also be added as needed.
Because every time a Feed sends a message, Fetch has to notify many other Feeds. So in this open environment, it is easy to cause Dos attacks. Therefore, when the Feed sends information to Fetch, it needs to pay a Gas fee. Only after receiving the Gas fee will Fetch put the information into the “pending notification” list.
The Gas fees for posting, liking, and commenting are 100000000, 10000000, and 1000000 respectively.
However, these are plans for the future. For now, let’s design the overall framework first and continue to optimize the details later.
Okay, after saying so much, let me explain this architecture in detail now.
Architecture
The architecture is based on a distributed peer-to-peer Actor model with a push-pull design. Overall, you can divide Proton into four modules: User, Feed, Post, and Fetch.
-
User: The user area, responsible for recording user information and relationships. It stores users’ profiles and follow relationships.
-
Post: The public area, storing all publicly posted content. The Root Post can create many Buckets to store posts.
-
Feed: The information feed, storing personal feeds for each user. The Root Feed creates a Feed for every user.
-
Fetch: The transit station, responsible for pushing the latest feed to each user. It records the unpulled posts, comments, and likes in a user’s Feed.
Users can follow/unfollow others, view the latest public posts (from everyone), see their feed (their own and followed users’ posts), create posts, repost, comment, like, favorite/unfavorite.
User
The user area records user information and relationships, like profiles and follow relationships.
The User canister stores basic user info - UserId, name, company, school, occupation, bio, follow relationships, their Feed canister ID, etc.
Users can call functions here to follow someone, update their profile, or check who they follow and someone else’s relationships.
When a user follows someone new or gets a new follower, their Feed is notified to update the list.
Post
The public area stores all publicly posted content. The Root Post can create many Bucket canisters to store posts.
Root Post
The public area stores all public posts. The Root Post can create many Bucket canisters to store posts.
Root Post can create Buckets, check available Buckets, list all Buckets, and list full Buckets.
Root Post starts by creating 5 Buckets. When one fills up, it creates a new one, keeping 5 available.
When the frontend first loads, the Feed immediately queries Root Post for an available Bucket. Root Post randomly returns one. The Feed stores the “available Bucket” ID, updating it when queried.
- Call Bucket to query the latest 5 posts to show latest public posts.
- When a user posts, call Bucket to store it.
When a user’s Feed gets a bunch of post IDs from Fetch, it can then provide those post IDs to the Bucket to query for the actual posts.
Bucket
Buckets can store and query posts.
There are 3 query functions - total posts, get specific posts by ID (up to 7 at once), and latest n posts.
When querying specific or latest posts, it returns post details and current likes/comments.
Buckets receive new posts, comments, and likes. It checks for duplicate IDs before accepting a post.
It notifies Comment Fetch and Like Fetch about post IDs with new comments/likes.
Feed
The information feed stores personal feeds for each user. The Root Feed creates a Feed for every user.
Root Feed
Root Feed is responsible for creating a personal canister for each user and tracking the total canisters created and their IDs.
Feed
Users interact with Proton through their Feed - viewing, posting, commenting, liking, etc.
A user’s Feed stores their followers (for pushing posts, comments, likes), following (for receiving posts), feed (last 3000 posts only), and saved posts (up to 500 posts).
Each post has a timestamp, poster’s UserId, PostId, and RepostId (empty if not a repost).
The PostId is the Bucket canister ID + UserId + increment. This allows direct post ID creation without communicating with the Bucket.
For example: aaaaa-aaa-bbbbb-bbb-1, aaaaa-aaa-bbbbb-bbb-2, aaaaa-aaa-bbbbb-bbb-3…
Querying posts:
There are 3 functions to query Feed posts - total posts, get post by ID, and latest n posts.
Posting:
When user A posts, the frontend sends the post to their Feed.
The Feed stores the new post.
Then it sends the post to the public Bucket and notifies Fetch with the poster, post ID, and followers C & D.
Fetch records this and individually notifies C & D’s Feeds to pull the post by ID. After notifying, Fetch deletes the records.
When Feed gets the post IDs to pull, it adds the corresponding posts from the public Bucket into the feed stream. (Here, User C pulls posts 1, 6, 7, 15)
When Users C & D query their Feed, they see Poster A’s new post right away.
If User E later follows A, their Feed only receives A’s new posts going forward.
The frontend only sends one request. Further pushes (like notifying the public Bucket) are handled by the canisters.
With more users, one Fetch may get overwhelmed with posts. More Fetches can be added horizontally as needed.
Deleting posts:
Posts cannot be deleted. The blockchain is immutable.
Older posts past the last 3000 are dropped from feeds anyway. (And Feeds are personal spaces).
Reposting:
When User C reposts Post 15 to followers H, I, J, K: Reposter: C, Post ID: post15_id, Followers: H, I, J, K is sent to Fetch.
Fetch records and notifies H, I, J, K’s Feeds, which pull Post 15 by ID.
When User C reposts, the original poster A remains the same. Only the reposter is User C.
Commenting:
Similar to posting, handled by Comment Fetch.
Anyone (User X) can comment. The public Bucket is notified of the comment with the post ID.
Comment Fetch gets the post creator’s followers from User. It rejects if the creator isn’t found.
It adds the post ID, creator, and followers to its “to notify” queue.
After Comment Fetch notifies followers, they query the post’s comments from the public Bucket and update their Feeds.
If a follower D reposted the post, their Feed further notifies Comment Fetch when receiving a new comment.
Comments cannot be deleted.
Liking:
Similar to posting, handled by Like Fetch.
Anyone (User X) can like a post. The Bucket notifies Like Fetch of the new like with the post ID.
Like Fetch gets the post creator’s followers from User. It rejects if the creator isn’t found.
It adds the post ID, creator, and followers to its “to notify” queue.
After Like Fetch notifies followers, they update the post’s like count in their Feeds.
If a follower D reposted the post, their Feed further notifies Like Fetch about new likes.
Likes cannot be deleted.
Fetch
Receives all posting, commenting, and liking notifications and forwards them to relevant Feeds.
Root Fetch
Root Fetch dynamically creates multiple Fetch canisters - Post Fetch, Like Fetch, Comment Fetch. It can also list available Fetches.
Post Fetch
Receives: Post ID, poster, reposter, followers, cycles.
Maintains a table of posts to notify each user about.
Uses an algorithm with ignore calls to notify followers’ Feeds in batches.
Comment Fetch
Receives new comment notifications from Buckets: Post ID, poster (A), reposter (empty).
Gets the poster’s followers from User.
Maintains a table of posts to notify about.
Uses an algorithm with ignore calls to notify followers’ Feeds in batches.
If a follower C reposted the post, their Feed further notifies Comment Fetch when receiving a new comment.
Like Fetch
Similar to Comment Fetch, for likes instead of comments.
The core User, Post, Fetch, and Feed modules make up Proton’s architecture.
Beyond this, Feeds can communicate directly for more features…
Building
This is a complex, large-scale application.
In this tutorial, we’ll build the core functionality of this Dapp.
First, We …
mean I will lose all my post published before the 3000th from the end?
, but it’s still in demo mode.