Deprecating the Service Worker

Hello everyone!

As some of you may already seen in our latest Global R&D, on Twitter or our latest dev blog, DFINITY is proposing to deprecate the Service Worker. We believe this is a huge step forward in improving everyone’s experience as they use Internet Computer dapps and it’s something that I am personally very excited for!

With a full rollout of this change, we don’t expect any impact to existing dapps. Dapps that currently run on ICX Proxy (raw URLs) will continue to function as normal. Those that are being served through the Service Worker will switch to ICX Proxy. We will have a canary release period that will help to identify any kinks and give us all some time to iron them out before we do a full release.

There’s a lot of reasoning behind this move. Too much to go into detail with in a forum post, so here’s a summary of those reasons and an overview of our proposed short-term roadmap. For anyone that wants to read more, the blog post has much more detail. If there are any concerns, comments, rants, musings or any other kind of relevant brain dumps, please feel free to comment here and we can discuss!

The motivation for this proposed change

User experience

  • Performance
  • Consistent branding
  • Social media sharing
  • Privacy focused browsers
  • Sharing assets on a web2 application

Developer experience

  • Enable developers to create progressive web apps
  • Lighthouse support
  • Content encoding (Brotli)
  • Caching
  • Remove necessity to bypass the Service Worker

Complexity

  • Remove SEO and /_/raw routing
  • No technology misuse
  • Decentralized API Boundary Node routing/discovery

The short term roadmap for this proposed change

The short term roadmap that we’re proposing is as follows:

Research and investigation

Timeline: From present day up to 2-3 weeks.

This is the current phase that we are in now. We are currently working on gathering metrics for the size of streamed response bodies. This is important because these responses are currently not verified by ICX Proxy. We would suggest gathering these metrics for the next couple of weeks in order to make an educated decision about the next step of the transition.

Limited certified response body streaming

Timeline: 1 week.

Based on metrics of typical streamed response body sizes, we will implement verification for streamed response bodies up to a safe limit. This limit should be high enough that it will cover JS, HTML, CSS files and average sized images, but low enough that it would not allow a malicious canister to perform a DOS attack on ICX Proxy.

This limit will initially apply to both response verification v1 and v2, but in the future we plan to develop a streaming protocol exclusively for response verification v2 that will allow for a much higher limit of streaming without exposing ICX Proxy to DOS attacks from malicious canisters.

Blocking of dangerous features

Timeline: 1 week.

Cache headers and redirect (3xx) status codes control browser behavior. The status code and response headers are not certified for response verification v1 so these should be filtered (headers) or blocked (status code) if a canister tries to use them. A canister should use the “upgrade to update” call feature if it wishes to leverage these features.

For response verification v2, the same should apply for cache headers only if they are excluded from certification, if they are included in certification (and the verification succeeds) then they can be passed along as normal. The status code is always certified so there’s no need for any blocking there.

Canary release

Timeline: 1 - 2 weeks, depending on the success of the canary release.

A canary release of the boundary nodes will direct all dapps to ICX Proxy instead of the Service Worker, which allows community members and dapp developers to test out their dapp through ICX Proxy and make sure that everything is working as intended. This will be a much more important canary release than usual and community support with this release will be vital for a smooth transition.

Full release

Timeline: Immediately following the successful canary release.

After a testing period with the canary release, assuming everything goes to plan, then this canary release will be rolled out to all boundary nodes, region by region and the Service Worker will be officially deprecated. This rollout will be different from standard Boundary Node rollouts. Usually the Boundary Nodes are enabled for a small percentage of traffic within a region and this percentage is increased gradually. Due to the nature of this change though, the usual approach will create a bad user experience if they are toggling back and forth between a service worker enabled boundary node and an ICX proxy enabled boundary node.

18 Likes

Is there any news on this, is it live now?

It’s not live yet. We’ve been preparing the code changes and are now creating tests to verify those changes. We will be aiming to go live with this in the coming weeks. I don’t want to over promise on the timeline because we want to make sure we take the time to ensure that things work and won’t be affected by the transition.

3 Likes

Understood.

The service worker has been keeping me from adopting the IC for general web hosting needs, for reasons outlined above. This is really important stuff, good luck with it.

2 Likes

Any updates on progress?

We are looking to start rolling this out this week so you can expect a public announcement in the next days :slight_smile:

11 Likes

Hello everyone,

I’m happy to announce that we’re ready to deprecate the service worker!

On Thursday Dec 7th, 2023 we will gradually start rolling out a new version of the boundary nodes that leverages icx-proxy to verify HTTP responses on the boundary node itself.

We’re excited for this change for all the reasons mentioned in the original post. Based on community responses, we believe it will make the user and developer experience that much more enjoyable.

Who is affected:

All the dapps being served from the DFINITY gateways (*.ic0.app, *.icp0.io and hosted custom domains) will be affected.

Note that dapps being served from a custom gateway implementation will remain unchanged.

What to expect:

The most noticeable change will be that the service worker loading screen will not pop up anymore. Inspecting requests in your browser console will show regular HTTP calls for fetching HTTP assets instead of the encoded ones.

It’s important to note that streamed responses larger than 8Mb will not be verified by icx-proxy and instead served as is. According to our metrics, requests with larger responses were very infrequent and represented an almost insignificant percentage of the total HTTP traffic. We will address verification for larger streamed responses at a later stage.

As a reminder, for those of you looking for an implementation of the HTTP Gateway Protocol that runs locally, you’re welcome to try out the IC http-proxy. In the future we will be investing more resources into the IC http-proxy to make it more efficient and extend support to mobile devices.

As usual, your feedback and comments are welcome.

23 Likes

I’m itching to see how the lighthouse scores stack up for websites hosted on the IC and accessed without the service worker.

Good performance is crucial for SEO and therefore also mainstream adoption.

4 Likes

Me too!

Just be weary of comparing the scores. Lighthouse can be a bit funny… Sometimes it measures the performance of the Service Worker alone (which is very good because it’s a single, tiny page) and sometimes it measures the performance of the Service Worker + Dapp.

So if you’re comparing Dapp alone to Service Worker + Dapp then the scores should improve,
but if you’re comparing Dapp alone to Service Worker alone then the scores may actually drop.

1 Like

Understood.

To be honest, my main interest is to compare websites hosted on the IC with websites hosted on the big Web 2.0 platforms like Netlify.

I am really interested in making a business case for moving client marketing sites onto the IC if possible and these comparisons are crucial.

I am already able to make a case in terms of sovereignty and security. I just need to know where we stand with performance.

Comparing Netlify and the IC’s boundary nodes isn’t a fair comparison and I don’t think you’ll see very positive results there. Netlify is a huge content distribution system with nodes in places we can’t even think to go yet with the boundary nodes.

It is possible to host your own HTTP Gateway with serverless functions though. I was experimenting a while back with hosting one on Cloudflare, it worked very well and that would be a much more accurate comparison.

4 Likes

That is very interesting.

What’s to stop a third party from providing an additional, globally distributed network of HTTP Gateways just to speed up performance?

I could imagine handling the set up for clients who wanted improved performance in a specific region for example.

The harsh reality for me is that as soon as I start hosting client websites on the IC, lighthouse will come in to play. All of the benefits of the blockchain will be disregarded if the technical SEO suffers.

2 Likes

What’s to stop a third party from providing an additional, globally distributed network of HTTP Gateways just to speed up performance?

Absolutely nothing! If you plan to do this, please do reach out as I’ll be working on similar things early next year and we could learn from each other’s experiences.

All of the benefits of the blockchain will be disregarded if the technical SEO suffers.

With a CDN/serverless-backed HTTP Gateway implementation, I don’t predict any issues with SEO. SEO crawlers won’t know any difference from a web2 site.

2 Likes

Understood. It’s more the performance metrics that concern me, but this conversation has been really interesting and has provided me with much food for thought.

I think that working on ways to boost the performance of websites hosted on the IC is a very interesting proposition. Thanks!

3 Likes

Hey everyone,

we have just completed the rollout of the boundary node release that deprecates the service worker. We deployed the release in two stages: Yesterday, we started with the European region. After 24h of “baking”, we continued the rollout today with all the other regions.

If you experience any problems, please reach out to us here on the forum!

6 Likes

hi,
I wonder what do I need to update in the project configuration with the latest updates - service workers depreciation, when visiting canister through the domain it doesn’t work, canister itself is fine and deployed. I have the before-update configuration from the docs: Custom domains | Internet Computer

update: resolved - seems like there were conflicting _acme-challenge TXT records in DNS settings

1 Like

Hi @tomkoom

Sorry for the late response. The deprecation of the service worker hasn’t changed anything for how you need to configure your custom domain. The only thing that changed is how your domain behaves once it is configured.

The most common problems are as you said conflicting _acme-challenge records due to your DNS provider also trying to get a certificate or missing configurations.

1 Like

Hi @NathanosDev , being able to use one’s own service worker is a great opportunity I think and I’d like to use it to turn my dapp/s into Progressive Web (d)Apps (as also mentioned as a motivation in your original post here). Do you have any good examples or resources how to do this for an app built on the IC? In particular, I’d be interested to enable offline functionality via the service worker (I’ve made the first PWA step, by adding the manifest and making it installable).

Some additional background: the DeAI chat app (https://x6occ-biaaa-aaaai-acqzq-cai.icp0.io/) I’ve been working on now also supports running the LLM on the user’s Android device (Chrome Canary, soon standard Chrome). Now, I’d like to use the downloaded LLM from the cache also if the device is offline via the service worker such that the user can chat with it nevertheless. I’d also be happy if you could share any general challenges you might see with how the service worker would be used here :slight_smile:

Awesome! This is one of the use cases that we wanted to enable by getting rid of the service worker, so I’m glad to see developers starting to do that.

Since the IC service worker was only deprecated recently, there’s probably not many examples around. The only one that I’m personally aware of is OpenChat, but I believe it’s only handling notifications.

In general though, doing this for a frontend on the IC is no different than it would be for a traditional frontend side. The only difference is using agent-js to communicate instead of fetch.

In terms of challenges, the biggest challenge I’ve encountered with service workers, both on the IC and in traditional websites is inconsistent service worker behaviour across browsers and not having a killswitch for a service worker. Without a killswitch a service worker can be stuck indefinitely on an end user’s computer.

I’d recommend checking out some pre-built solutions like Google Workbox or something similar, to make your life easire.

2 Likes

great, thanks a lot for sharing the links and info! Excited to implement it now and see how it works :slight_smile: