Creating a Faster Jekyll

Jekyll is a static site generator which we recently adopted for most of https://macromates.com motivated by its nice design and large userbase.

We did however run into performance issues so we wrote a replacement which is semi-compatible with Jekyll but with better speed and some additional features

Feature Summary

• Running serve will generate content as requested by the browser (lazy evaluation), this allows instant previews as the full site doesn’t have to be built first. It also means that if there is a (syntax) error generating a page, the error will be shown in your browser and the error page even supports automatic reload. Another advantage of this approach is that during testing we do not write anything to disk, so _site will always contain deployment-ready pages (no instances of localhost:4000, injected reload scripts, or unpublished drafts).

• Running build will make use of multiple tasks to parallelize content generation. Our site took almost 10 seconds to build with Jekyll which we have now reduced to 2-3 seconds (this is on a 2.8 GHz Macbook Pro from late 2013).

• Collections have been generalized so that they all support both tags, categories, drafts, and arbitrary sorting (e.g. reverse chronological). There is nothing special about _posts.

• Support for multiple domains has been added. This means generating content for example.org and blog.example.org can be done using the same project, so that resources can be shared and cross-linking is possible via the link tag.

• Extensible render pipeline: Content is transformed using a pipeline where it is trivial to add new filters, this allows adding new converters, override the default converters, or simply pre/post-process content to support custom syntax, inject content, run the generated HTML through a validator, or similar.

• Introduced a digest variable which can be used in permalinks to ensure that a page’s URL will change when the content is updated (guaranteed cache invalidation useful for CSS and JavaScript).

• Easy pagination of both collections and data structures.

• Collections can have pages generated for tags and categories. Making this a built-in feature makes it possible to iterate generated pages and link to these using their url property rather than make assumptions about where such pages end up in the file hierarchy.

• Any change to a site file, be it files under _data or even _config.yml, will trigger a browser reload that will fetch the updated page. This is possible because we use lazy evaluation, so a file system change is effectively just triggering a cache flush, rather than having to rebuild the entire site.

• Default values for pages can be set using file globs, making it easy to use the same set of values for a broad set of files, and default values for collection files can be set under the respective collection, which is extra useful when using cascading configuration files.

• Introduced a source_dir setting to allow putting site content in a subdirectory, obviating the need for maintaining a list of excludes and/or prefixing non-publishable items with underscores.

Getting Started

Since this was started as a reimplementation of Jekyll, let’s use Jekyll to build the initial site scaffolding:

jekyll new awesome-site

Add Glim to the generated Gemfile:

cd awesome-site/
bundle add glim

And then test the site:

bundle exec glim serve --open-url

If you are using TextMate then install the glim-edit-in-textmate plugin like this:

bundle add glim-edit-in-textmate --group=glim_plugins

Re-run the serve command and in your browser you will be able to press e to open the source for the current page in TextMate.

This uses the txmt: URL scheme so your browser will ask for permission (the first time, depending on browser).

If you don’t want to use a Gemfile then you can install glim using the gem command:

gem install glim

Run glim help to see what commands are available.

Detailed information about the syntax for the features mentioned above can be found in the glim manual.

Source Code

The source for Glim is available on GitHub.

Future

While compatibility with Jekyll was a goal for the first version of Glim, this is not a long-term goal, and we do consider breaking some of the existing compatibility.

For example with Jekyll, using :title in a permalink, page.title in a template, or post.title when iterating over posts, can result in 3 different values for the same page. This inconsistency adds code complexity and makes it harder for the user to understand the rules.

Another example is the API names, there are getters for pages, documents, posts, files, and docs. This again is inconsistency that makes it hard for the user to understand what type of files are returned by the respective getters.

Since we cannot offer 100% compatibility with Jekyll, we may as well make a clean break and clean up these things.