Is it possible to make a search engine in Dfinity?

I was watching the CanCan video on this page:

, and it strikes me, that - if it’s possible to “shard” data across multiple servers, and send it back to the user, then is it possible to store an INDEX, as used in a search engine, Google style, and have a text search run on it and send data back to the user? That would obsolete Google, wouldn’t it?

Anyone got any ideas/thoughts on this?

BigMap has some sort of search functionality built in, will be interesting to play around with.

Haha… unlikely.

4 Likes

There is a lot of work there. The automatics behind googles robots is complicated (considering the fact we don’t have SSR in the IC).

But, one can easily imagine some kind of community driven platform, like an app catalog, where everyone could register their app with some metadata (e.g. IT articles for beginners or spanish cuisine cookbook) and all community members could validate that “yes, this is indeed a spanish cuisine cookbook site” in a convenient manner for some small incentive.

The only pitfall here is the identity management (sybil attack). And this is actually a very good task to implement a solution for - almost any community-driven software needs one of these. This could be solved with typical “I have a tokens so I can vote” solution, but this solution is not ideal, IMHO.

1 Like

The Motoko implementation of BigSearch can be found here - I think this is a great start as well!

1 Like

The bigsearch repo does not seem to contain any code…?

Oops, my fault.

The code was (originally) straddling two repos, but I ended up just moving it into the repo here:

Sorry for the confusion.

4 Likes

There is also some Rust code that implements an enhanced Rust-based BigMap / BigSearch.

The code above is a Motoko (only) effort.

Y’all might like this talk from last Autumn in Prague:

“Decentralised Hashtag Search and Subscription in Federated Social Networks” – Schmittlauch

Sorry my head is in the way in the bottom right!

2 Likes