Now that the IC supports crawlers (I’m so pumped about this 
), I wrote a guide to help other developers prepare their dapps for social media sites and SEO.
If you have any idea of improvements, please let me know 
10 Likes
Just done reading it and I love it, I did bookmark it too for later reference, thank you spider man.
1 Like
haha cool to hear that, my pleasure 
1 Like
On 
as always
Would be cool if you could publish the content also at Deploying & Upgrading Canisters | Internet Computer Home or Best Practices | Internet Computer Home
to make it easier for future devs to find it. What do you think?
2 Likes
If you think it’s an interesting information for the docs, sure it’s cool 
.
It probably better fits “Best Practices” I would say. For sure it would also deserve a good review first (my english is notably not the best).
P.S.: the website supports canonical? not against keeping Papyrs as original source but not a must neither
1 Like
good stuff. my dapp uses the ic raw link. do u know if one can use a custom domain. it is making it hard to do any seo
It’s possible to use a custom domain (see documentation below) but note that I think this new feature of the issue - support for crawlers - does not work with custom domain.
Custom Domains | Internet Computer Home & The Internet Identity Specification | Internet Computer Home
Note that if you use a custom domain you may need to serve a custom worker but, since you are already using the raw domain I guess you are fine not using one.
Alternatively, there is also https://fleek.co who offer an easy solution for hosting and custom-domain but note that with this solution you are not directly the controller of the canister anymore.
1 Like
Hey can you point me to why you’re thinking support for custom domains and crawlers do not work together? I’m currently interested in doing both so this would be great info to help make those choices.
It seems like the boundary node team has custom domains in the works, I imagine their solution will be compatable. Boundary Node Roadmap
Not sure anymore if I heard that in a presentation or if it was an input from @domwoe. Dominic do you know a bit more, can you answer?
1 Like
Hi Mitch, in order to support crawlers, you must bypass the service worker. canister-id.raw.ic0.app, for example, does that. This bypass is now also enabled for the standard canister-id.ic0.app domain. We achieve that by checking the user-agent and redirecting crawler requests to “raw” instead of serving the service worker.
If you follow the current custom domains instructions, you set up your own frontend that serves the service worker. In order to enable crawlers + custom domains, you would have to detect crawlers in your frontend and then either serve the service worker or redirect the requests to “raw” yourself. It’s certainly possible, but more involved.
Once we support custom domains on the boundary nodes directly, crawlers + custom domains will also work.
3 Likes
Ah great news, thank you for the reply, that clears things up. I’m looking forward to custom domains out of the box!
1 Like
Can I still find this blog somewhere? The canister doesn’t exist and Papyrs seems to have been shutdown 