Using GitHub webhooks to rebuild static web sites

(Originally published in 2019.)

GitHub’s webhooks are wonderful things: with a little bit of configuration, you can be notified whenever your repository has new code pushed to it.

This web site is powered by Jekyll, a static site generator. My build process for it basically looks like this:

ssh markchristian.org
cd /home/mark/writing.markchristian.org
git pull origin master
bundler install
jekyll build

It’s not particularly onerous, but it’d be nice if it just happened automatically whenever I pushed a change. As a hypothetical future bonus feature, if all I had to do was commit to the repository, it’d be a lot easier to update the site from my iOS devices.

Receiving webhooks

Webhooks are just HTTP requests, made to a URL that you specify. I didn’t want or need anything complicated from the webhook—just a little nudge to let my server know that a change had occurred. I’m using DreamHost, and the path of least resistance there is to use PHP. So, I started out with a webhook that looked like this:

<?php system("/home/mark/bin/pull-and-update-site.sh"); ?>

You’ll note that this has absolutely zero validation to make sure that the request actually came from GitHub. If this was a more serious integration that actually did something with the incoming information, I’d actually secure my webhook, but since the only information we want is “something happened”, it’s fine without it.

Unfortunately, this first implementation didn’t work, and for an obvious security reason: web requests aren’t handled by the same user account that I log in with. Of course we don’t want our Apache process to be able to access my SSH keys (to update the local checkout of the git repo) or update Ruby gems. The question was: what to do? Something needed to be able to do those things in order for this to work.

Updating a status file

One thing my PHP endpoint could do was write a file to my user account. So, I updated my webhook:

<?php file_put_contents("/home/mark/webhook-was-called", date()); ?>

Now, whenever the webhook was hit, it’d write to a file called /home/mark/webhook-was-called. It doesn’t matter what, if anything, you write, or even whether it’s different than what’s already in there, but I figured that knowing the timestamp of the last webhook call might come in handy someday.

The next step was to create something that ran within my user account and watched that file for changes.

Watching for changes

There are a few ways of watching files for changes, but none of them were quite as simple as what I wanted. I ended up creating a simple new tool called watchnrun, which is a very simple command-line wrapper around the excellent listen Ruby gem. It’s used like this:

$ watchnrun /home/mark/webhook-was-called /home/mark/bin/pull-and-update-site.sh

At this point, everything did exactly what I wanted:

I’d push a change to this site’s GitHub repository
GitHub would make a request to my webhook
My webhook would write to /home/mark/webhook-was-called
watchnrun would run /home/mark/bin/pull-and-update-site.sh

The last problem was that I needed to have watchnrun running all the time.

Running watchnrun in the background with screen

If I was using on a dedicated server rather than DreamHost’s shared hosting, I could’ve configured this to run at startup automatically, but DreamHost’s servers are (rightfully) pretty locked down. I needed an alternative. Fortunately, the server I’m on restarts very rarely, so having to manually start it up wouldn’t be the end of the world.

screen gave me the last piece of the puzzle. For those who are not familiar with screen, it allows you to start a terminal session and “detach” it, leaving it running in the background even if you end your terminal session. It was just what I needed.

I started watchnrun in screen like this:

screen -dmS hooks-writing-watcher /home/mark/bin/watchnrun /home/mark/webhook-was-called /home/mark/bin/pull-and-update-site.sh

The -d tells screen to detach at startup. The m tells it always create a new screen session, even if there’s already one running. The S tells it to give that screen the name hooks-writing-watcher, so I can re-attach to that session if I want to by typing screen -x hooks-writing-watcher.

It worked!