I recently realized something that made me truly realize how powerful service workers can be. When I have internet, it feels like there are infinitely many things competing for my attention. But when I am on a plane for example, and there is no internet connection, the competition for my attention is much less fierce. The 3 things I can usually do is look through my photos, watch a downloaded movie or read an ebook. With service-workers, if you are able to deliver an offline web experience for your users, you are able to get their attention in one of those few moments when the competition for it is least fierce.

After reading this article you will learn, the theory of how service workers work, a short tutorial to apply that theory to make a website that runs without the internet and finally, what this means for the you and future of the internet.

I get really excited when talking about service workers so I am going to go on a quick rant on the problem this solves and why this is awesome before I dive into the theory of how service workers work and a coding tutorial. If you want to jump straight to the theory or the tutorial, just go to the sections starting with titles Theory and the Application section for the code.

 

This blog post is also available as:

Table of Contents

  • Websites are Strange

  • Theory: How Servicer Workers Work

  • Application: Tutorial on How to Build Offline Websites

  • Prerequisites

  • Installing the Service Worker

  • Part 1a: Build The Service Worker

    • Service Worker Output

  • Part 1b: Testing The Service Worker (#2001441)

    • Creating A Mini Server

    • Inspecting the Server Requests

    • Where are the Files Being Saved?

  • Part 2: Saving External Data (Part 1 Git Tag: pwa-tutorial-0.1 )

    • Saving External API calls: #8593ada

  • Part 3: Notify Users of New Updates (Part 2 Git Tag: pwa-tutorial-0.2)

  • Part 4: Deployment (Part 3 Git Tag: pwa-tutorial-0.3)

  • Conclusion

    • Who Needs Mobile Apps

    • The Future of Websites

  • Further Reading

Websites are Strange

Let’s start with a simple diagram. What do the two circles in this venn diagram represent?

Let me give you a hint. Amazon, Alibaba, and Facebook are some of the world’s biggest websites, servicing millions of users everyday. Here are some stats to put things in context for you:

  • Alibaba $25 billion in sales in one day (singles day)

  • 40% of cloud computing clients use Amazon Web Services Including: Apple, Netflix and CIA

  • 2.2 Billion people use Facebook every month, 700 million on Instagram

This is all well and good but there’s just one small problem. Without wifi their entire websites are completely unusable. Even if you just want to do simple tasks like look at product reviews for items that are already in your shopping cart, you can’t do anything.

Now compare that to sites like Google Drive or Atila.ca. Atila.ca doesn’t have a million users but even when you have no internet connection, you can still use the site. Google Drive is another website that does this well. You can actually use google drive even without internet, similar to how you would use a desktop app like Microsoft Word. Learn something new everyday right?

The Venn Diagram labelled

In the past it almost seemed like a truism that websites would not work without the internet. Once you realize the potential of service workers, you completely change your perspective on how you think about websites. You begin to imagine the significant user experience improvements we can have on our favorite sites.

Imagine if you are on the subway commuting to work with no internet and not even any cell phone service but you could still review the product reviews of the items in your Amazon shopping cart. Or you’re on a long plane ride and while your phone is in airplane mode you can read the most popular articles from The New York Times or your favorite articles in a list that you chose to save for later.

You can see that the potential is great and it easy to start daydreaming. Let’s get back to reality and dig into the theory of how this is all possible.

Theory: How Servicer Workers Work

A service worker is essentially just a proxy or messenger between your browser and the internet. When your web app asks for resources (images, html files, json API etc.) the service worker gets it for you without needing to ask the internet. In literal terms, it is a javascript file that gets shipped along with the rest of your app with code that tells your app how to intercept network requests and save and get it from your network cache.

Typically, when website is first loaded, the web browser makes a set of requests to the network for the assets that your website needs to run. Typically including:

  • html files for content display

  • css files for styling

  • Js files for application logic

  • Images and other assets

When your internet is not working. There is no way for the browser to contact the network to retrieve the files necessary for displaying the website so it fails and you get the infamous “jumping Dinosaur” on Google Chrome.

With service workers. The first time you visit a site, in addition to the usual set of index.html, styles.css, main.js etc. files that get requested, the browser also requests a service worker javascript file from your website. This file then gets downloaded and saved to your browser cache. The serviceworker file then downloads, versions and caches all the files in your application to serve it for later.

This has two very important benefits. As is mentioned in the title, this means that even when your internet connection is down, the website will still work because it never actually has to ask the internet for anything. The service worker, has all the files you will need. Conversely, this also means that even if you do have internet, the application will also run faster. On future website loads, instead of making a full round trip to the network to get the application files. It simply retrieves the files from the browser cache and serves it to the user.

Application: Tutorial on How to Build Offline Websites

Prerequisites

Before you get started make sure you have the following installed

  • Node.js® and npm

  • Make sure you have node 8.X or greater (node -v ) and npm 5.x or greater ( npm -v)

  • Globally install Angular CLI:

  • npm install -g @angular/cli

  • Google Chrome Browser (optional, but recommended)

  • Google Account (optional, only if you want to deploy to Firebase)

Launch the project

If you get a No NgModule error. Go to any .ts file and put a space. This is a really strange bug but you can read this Github issue for more information.

Installing the Service Worker

In order to install the Service Worker and get access to the various Service Worker objects in our app we need to do the following.

  1. Install the @angular/serviceworker module: npm install @angular/serviceworker

  2. This installs an npm package that includes various service worker javascript objects that we will use in the future steps

  3. Tell angular-cli to build project with service worker: ng set apps.0.serviceWorker=true

  4. Tell angular CLI to automatically produce a javascript file that contains the code for the service worker, when building project explained in next step.

  5. Configure your service worker in ngsw-config.json

  6. This tells your service worker, exactly what files it should be saving and how it should be saving them:

  7. assetGroups: files that are included as part of the app,

  8. When the app updates, these resources update as well

  9. dataGroups: external resources not versioned with app

  10. Install Mode: What caching strategy to use when first seeing this file

  11. UpdateMode: What caching strategy to use when updating the file, after we’ve already installed it

  12. Caching strategy:

  13. Prefetch: Save these files before we even ask for it

  14. Lazy: Save these files only after they’ve been requested at least once

  15. Add a manifest.json file and reference it in index.html

Service Worker Configuration

  • This tells your service worker, exactly what files it should be saving and how it should be saving them:

  • assetGroups: files that are included as part of the app,

  • When the app updates, these resources update as well

  • dataGroups: external resources not versioned with app

  • Install Mode: What caching strategy to use when first seeing this file

  • UpdateMode: What caching strategy to use when updating the file, after we’ve already installed it

  • Caching strategy:

  • Prefetch: Save these files before we even ask for it

  • Lazy: Save these files only after they’ve been requested at least once

  • You can check out the official angular documentation on servicer worker configuration for more details

Manifest.json

{
"name": "Atila",
"short_name": "Atila",
"start_url": "index.html",
"display": "standalone",
"icons": [{
"src": "assets/img/favicon-bg.png",
"sizes": "512x512",
"type": "image/png"
}],
"background_color": "#194f87",
"theme_color": "#194f87"
}

This is what turns your app from a web app to a progressive web app. It allows your web app to be more similiar to a native mobile. It allows users to install your app to their app home screen.

Register the Service Worker (#d2b186f)

Now we need to actually tell our app that a service worker exists. So we register the service worker module in our app module.

// src/app.module.ts

import {ServiceWorkerModule} from '@angular/service-worker';

…

Imports: [

…,

ServiceWorkerModule.register('/ngsw-worker.js', {enabled: environment.production}),]

Then we register the service worker file if our browser has service worker supportand we are are in production mode.

// src/main.ts

if ('serviceWorker'  in  navigator  && environment.production) {

console.log("Service Worker in main.ts");

window.addEventListener('load', () => {

console.log("on page Load Service Worker in main.ts");

navigator.serviceWorker.register('/ngsw-worker.js', {

scope: '/',

})

.then(registration  => {

console.log("Service Worker registration completed main.ts", registration);

});

});

Part 1a: Build The Service Worker

Service Worker Output

Next we will build the project: ng build --prod

Let’s take a look at the dist/ folder to see what building an app with service worker looks like.

Service Worker Output: ngsw.json

Recall that in the previous step we created a file called that ngsw-config.json that specified what types of files we wanted our service worker to cache and how we wanted to cache it. When the project gets built, the rules in the ngsw-config.json gets expanded to include exactly what files we will be caching. The ngsw.json file also includes a hash table to index and retrieve the cached files, the hash table also allows us to version our files so we can keep track of what versions of our file is running and if we should get a new version.

Service Worker Output: ngsw-worker.js

This file is literally the service worker. We registered it earlier in our main.ts file and it is a plain javascript file containing the code and logic for how your service worker registers and caches your service worker to database. If you’re up for a challenge, try looking through the code and see if you can understand what is happening.

Part 1b: Testing The Service Worker (#2001441)

Creating A Mini Server

  • Service workers are used in an offline contexts, so we need a server that can simulate offline environments

  • Install npm http server

  • Npm install http-server@0.11.1 --save-dev

  • Build and Run the server:

  • ng build --prod (optional)

  • http-server -p 8080 -c-1 dist

Inspecting the Server Requests

  • Visit Chrome Network tab in devtools

  • Do this before going to localhost!

  • Open a new tab

  • Right click somewhere blank on screen

  • Inspect > go to Network Tab

  • Open http://localhost:8080/

Note that there is no wi-fi in top right. Check the devtools console: The external network resources fail with 504 but our files are succesful (200).

Where are the Files Being Saved?

Open the application tab in Devtools and you will see local cache section. This is the “database” where the service workers are saving your files. I believe that there are 2 tables, one which contains the actual resources our app needs and another hash table with hash keys that point to each file name, just like we saw in our ngsw.json file. That’s it! You now have a simple but fully functional offline first web app. Continue to part 2 for adding more cool features.

Part 2: Saving External Data (Part 1 Git Tag: pwa-tutorial-0.1 )

Why are None of my APIs Working?

When you try to navigate click on a link, you will notice a server error. Your service worker doesn’t have those APIs in your database, but we can add it. Pop Quiz! If we want to tell our service worker to cache a new type of file where should we put the code to do so:

  1. Manifest.json

  2. Ngsw-config.json

  3. App.module.ts

You can see the network requests failing in your dev tools network tab.

Saving External API calls: Configuration (Github Diff)

  • Now we will configure our ngsw-config.json to cache external API urls as well. Two caching options

  1. Freshness: Go to the network first, if missing, go to the

  2. Performance: Go the cache first, then go to network

Saving External API calls: #8593ada

  • You might need to use a seperate URL that allows your app to access it via CORS. We will use a special JSON service to simulate (“mock”) our blog API. Since We don’t have permission to use official Atila API’s

  • Change ScholarshipService.getPaginatedScholarships:

  • Go to src/app/_service/scholarship.service.ts#L47

  • Change: this.http.post(${this.scholarshipsPreviewUrl}?page=${page}/, form_data)

  • To: this.http.get(https://api.myjson.com/bins/dx1dc)

  • Change BlogPostService.getBySlug:

  • Go to src/app/_service/blog-post.service.ts#L25

  • Change: this.http.get(${this.blogUrl}blog/${username}/${slug}/)

  • To: this.http.get(https://api.myjson.com/bins/v5ow0)

Part 3: Notify Users of New Updates (Part 2 Git Tag: pwa-tutorial-0.2))

  • When a new version is available from network, service worker still serves the old version in cache to save time

  • Add your profile to team page

  • src/app/team/team.component.ts

  • Add your image and some information in the team data array

  • If you rebuild the project and restart server, you’ll notice that your profile doesn’t appear yet.

  • We can add a snackbar to notify user of new updates

  • npm install @angular/material@5.1.1 --save (you might already have this)

  • Then we create swUpdate to listen for updates from the SW and update if a new version is available.Github Diff

// src/app/app.component.ts
    
import {SwUpdate} from  "@angular/service-worker";
...
    Constructor (..., public swUpdate SWUpdate,)
...
ngOnInit() {

if (true) {

// check service worker to see if new version of app is available

if (this.swUpdate.isEnabled) {

this.swUpdate.available.subscribe(() => {

const snackBarRef = this.snackBar.open('New version available', 'Load New Version');
snackBarRef.onAction().subscribe(

() => {

location.reload();
});
			});
		}
	}
}

Rebuild and reserve your application

Part 4: Deployment (Part 3 Git Tag: pwa-tutorial-0.3)

Deploying to Firebase Hosting

To truly see effect of service worker we should probably deploy it to a real website. I like Firebase Hosting because you can take a web app from your local host to a live website in less than 5 minutes with just a few simple steps. I’ve been doing this for almost 2 years now and I stil get impressed with how easy the process is.

Deployment Prerequisites:

  1. Create a google account

  2. Create A firebase account and a firebase project

  3. install firebase tools globally: npm install -g firebase-tools@4.2.0

  4. Log in to firebase: firebase login

  5. Initialize Firebase Repo: firebase init

  6. Choose the following settings:

    • Choose hosting as the project type

    • Change public folder to dist/

    • Configure as a single page app

    • Overwrite idnex.html? NO

  7. Deploy! Firebase deploy

  8. Visit the url in the command line output to see your app

That’s it! Congratulations you now have a website that can work without the internet.

Conclusion

Who Needs Mobile Apps

I’ve written previously about why I strongly believe that startups and companies should design their technology products as progressive web apps instead of mobile apps. My core arguments being:

  1. Having to maintain 2 separate Android and IOS codebases: 2 sets of users, 2 sets of developers, 2 sets of problems. (Things like Flutter and Ionic are cool but I don’t believe they address the fundamental issue of 2 seperate codebases)

  2. Most people only use the same 5 apps a day, convincing them to download another app will be very difficult.

  3. Approval and Distribution through Apple App Store and Google Play Store gatekeepers is not fun.

Some of the core arguments against web apps include things like:

  1. Offline accessibility

  2. Better user engagement because of push notifications

  3. Access to Native hardware features like cameras

With features like service workers I have just shown how we can enable websites to work offline, progressive web apps also enable push notifications.

Native hardware features is still the one advantage which mobile apps have, though this is changing. Unless your app is mission critical to have these native features I strongly encourage startups and companies to consider looking into making their next app a progressive web app. I think that the future of the internet will see more applications distributed through browsers and urls, than native apps and the app store. In an ironic case of history repeating itself, this is actually a throwback to the advent of the internet in the 90s when companies like Netscape and various “.com”startups” were doing very well distributing their software through the web.

The Future of Websites

Wouldn’t it be awesome if other sites applied that same philosophy: Imagine if you’re like me and you have a hour plus commute to work every day. Forget about wifi, they don’t even have cellular service on the subway.

Now suppose I want to buy bluetooth headphones and I want to read some Amazon reviews before I make my decision. Wouldn’t it be awesome if I could cache the product details for the items in my shopping cart and read the reviews on my way to work. Then by the time I get to work, I can just buy the item I want. Imagine you’re on the plane for a quick flight from Toronto to Ottawa. You load up the NY Times to catch up on morning news but the site gives the famous “Google dinosaur”. In an ideal world, the NY Times should cache the top 5 most popular articles and let users read the articles and leave comments offline that can be synced when they connect back to the internet.

Google Docs does this best. Google Docs is basically Microsoft Word, Powerpoint and Excel but better and built for Chrome. Obviously, you don’t need the internet to use Microsoft Word so just because Google Docs runs in a web browser, you should be able to use Google Docs to access and edit your recent items. Which is exactly what Google Docs does beautifully well.

I feel slightly proud of the fact that the great Amazon and NY Times websites need the internet to run, but humble little Atila.ca runs just fine with and without internet ;) When the internet is down, you can still see featured scholarships, blog posts and other pieces of content.

Okay, these are mostly #FirstWorldProblems. Another thing that gets me really excited about this technology, is building internet for the next billion users. People who have very slow or inconsistent internet connection.

For example, “HospitalRun is an offline first application for managing hospitals in the developing world, places where intermittent connectivity is just a fact of life. It allows records to be carried to remote clinics, where there may be no internet, and then syncs those records when there is.” (h/t jonbellah.com)

At Atila, one of our projects is increasing access to scholarships for students from developing countries. So it would be awesome if they could seamlessly read sample essays and work on their scholarship applications even if they lived in places with poor internet connection.

I personally find the idea of service workers and offline-first web applications very fascinating. Simple technologies with the ability to fundamentally change the day to day habits of billions of people are why I chose to study software engineering and why I love this field. I’m really excited to not just see how the future of the internet will unfold but also help build the future of the internet and how we will interact it. The future is already here folks, Thanks for reading.

Further Reading