Resource | Static Website Generators: Ultimate Caching?

Static Website Generators: Ultimate Caching?

By Alexandre Barret, 29 June 2017, 12 min read

A love story

Static files handle large traffic easily. Photo cred: Sergey Pesterev (Unsplash)

People generally see Static Website Generators (SWG) as solutions for blogs and company/marketing websites. They’ve, however, also been considered as an alternative way of building the web.

In this article, we’ll discuss the relevance of a SWG as a web application solution. A side project I worked on: (Bravo-Welly) will help demonstrate the points made. We’ll conclude with a possible way of building modern media or listing apps with a hybrid stack including a static website generator.

First, go have a look at the awesome project! Bravo Welly Wellington Festivals Made Simplebravo-welly.nz

Here is the technology behind Bravo-Welly:

Middleman (Static Website Generator)
Github pages (Free Static Hosting)
Turbolinks & Jquery (Javascript Libraries)
Google Maps & **Facebook SDK **(3rd Party Libraries)
Cloudflare (Content Delivery Network)

The main goal of the project was to learn how to create a website that feels fast, for cheap.

CHEAP

Static websites only use storage which costs nothing to host. Several services like Github even offer to host your files for free. In the end, this project could have been completely free if I did not decided to use a custom domain for it.

As a result the total price of that project was $29.79 (excluding dev time).

FAST

Caching is intimidating. It feels costly, difficult to get right and as a developer I have yet to code an application with a caching strategy. However if you are serious on speed then bite the bullet and just do it.

Middleman in my eyes represents the simplest caching solution. You can’t beat static files. It is reliable, easy to understand, fast to serve and easy to scale.

Load testing the app with 100 concurrent users during 6000 requests with a delay of 1 second per request shows no failed transactions.

`siege -r 20 -c 100 -d 1 -i -f ~/urls.txt`

Transactions:             6198 hits
Availability:             100.00 %
Elapsed time:             128.01 secs
Data transferred:         350.98 MB
Response time:            1.77 secs
Transaction rate:         48.42 trans/sec
Throughput:               2.74 MB/sec
Concurrency:              85.54
Successful transactions:  6198
Failed transactions:      0
Longest transaction:      8.83
Shortest transaction:     0.05

ab -n 6000 -c 100 https://bravo-welly.nz/list/

Concurrency Level:      100
Time taken for tests:   74.139 seconds
Complete requests:      6000
Failed requests:        0
Total transferred:      204727990 bytes
HTML transferred:       201612000 bytes
Requests per second:    80.93 [#/sec] (mean)
Time per request:       1235.651 [ms] (mean)
Time per request:       12.357 [ms] (mean, across all concurrent requests)
Transfer rate:          2696.69 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       55  549 493.2    458    5648
Processing:   241  663 235.1    633    3270
Waiting:      206  451 217.6    435    3094
Total:        338 1212 554.5   1117    6513

Turbolinks has a bad press among developers however I highly recommend using it if you are serious about performance. Applications like Basecamp, Github or Shopify have a rendering time below 100ms and all use a library like Turbolinks to achieve this. There is probably a reason. Check out Turbolinks *Abletecher Alexandre Barret has been using Turbolinks. Sit in on this Abletech Tech Talk to find out more.*stories.abletech.nz

Middleman + Turbolinks represents the simplest caching strategy possible to implement. You provide caching on both back and front ends. The website feels blasting fast.

Boom! Users are happy.

Cloudflare in this project was not really relevant as it was mainly used to get the SSL done on a custom domain. Github pages do not provide https for custom domains. In addition, Bravo-Welly is only relevant in New Zealand and a CDN was not necessary in this case. However a CDN is a must if you want to increase your response time for users around the world. Just get in the habit of doing it.

Finally, it is somehow poetic to think that you can create a simple application using only HTML, CSS and Javascript files. After all this is the web in its simplest form.

However using a Static Website Generator comes with its drawbacks

Build Time

The more content and pages you need to create the more time it will take to build the website. The build time will increase linearly with the number of pages to build.

Bravo-Welly is a small project and has only 153 files to build. On my local machine middleman builds them in 31 seconds which is approximately** 5 pages per second.**

If build time increases, switching to a faster generator like Hugo or implementing partial builds could be possible solutions. You do not need a complete build each time you deploy to production and you could select the files to create.

Dynamic Content

Displaying dynamic content with a static generator website seems counter intuitive; difficult but not impossible.

For example, in Bravo-Welly, each restaurant page has a review/comment section at the end of the page handled via the Facebook comments plugin. This has the benefits of delegating the social interaction between users to a third party that handles storing and displaying all the messages.

Also nothing stops you from performing XMLHttpRequest to get the dynamic content through an API and display it on the page.

Another idea is to use the Firebase service to access data that gets updated on the fly with websockets. Potentially, this could be an elegant solution for progressive web apps.**

Modification

Any modification to content that is supposed to be static is not trivial. There is no administration panel like with a Content Management System (CMS). It requires dev knowledge on how the data is used, where it is stored and sometimes how templating works.

In addition, Middleman does not have any partial builds and a small change in the data used to generate the pages triggers a complete rebuild which can be slow and annoying.

Why not have the best of both worlds: dynamic edition and static rendering?

I will conclude on what I think could be an interesting stack for apps with high traffic and a high number of pages with few updates after publication. I am thinking of media apps (online newspaper), event apps (Bravo-Welly), listing apps (Shop, Ebay like, Housing).

Considering the work done on Bravo-Welly, I imagine that the number of edits for each page is way lower than the number of views requested for that same page.

Dynamic Edition

Then, we could think of having a typical ‘administration app’ that manages the content required to create all the pages via a database. This ‘administration app’ would live on a subdomain and handle a low traffic of users willing to create and update pages just like a CMS.

For example, a journalist writing an article for an online newspaper, an agent creating a listing for a new house to sell or an event organiser on Bravo-Welly entering burger details for a restaurant.

Static Publishing

On the other hand, a SWG app would be responsible for building all the pages. To create those, the SWG app would access the same database as the ‘administration app’ and use that data in templates to generate the static files, store them and serve them.

This could be done in bulk updates or via scheduled jobs. Live updates might not be needed and a complete rebuild every 15 minutes could be good enough.

Benefits?

Scaling an application with more servers can become really expensive while storage is really cheap, easy to serve and performant on a high traffic of users. It feels more work overall but also more practical and reliable once the page generation is mastered.

Anyway

I am pretty sure the idea is not old and has already been used in some applications. The work done on Bravo-Welly made me think that this could actually work pretty well if the app had an ‘infinite’ number of festivals with an ‘infinite’ number of events to display .

I could not find any companies that explicitly use that system. If you do, please comment about it, I would be keen to see a practical example of that. There are, however, some people on the internet dedicated to make this happening with what is called JAMstack. If you are interested to see how static websites can change the way you program, have a look at those resources:

https://www.staticgen.com/: List of SWGs
https://www.quora.com/What-is-the-concept-behind-JAMstack
https://headlesscms.org/: A List of Content Management Systems for JAMstack Sites
https://jamstack.org/: Modern web development architecture based on client-side JavaScript, reusable APIs, and prebuilt Markup.

At the end it can also be unpractical and just easier to do caching right, but the idea feels interesting enough to me to write a post and share it with you.