My trick for "hot reloading"

A lot of the software I use embeds a scripting language for easy extension - this is great, but much of it doesn't provide facilties for "hot reloading"; that is, iterating on an extension without needing to restart the application and blow away its current state.

For example, I can define custom widgets and hooks in my window manager Awesome, but I need to restart it to see any changes I make while working (which, considering it's the bedrock of my desktop environment, always makes me 😬). I have techniques for reducing the blast radius if when I make a mistake, but the edit-restart-observe cycle isn't great.

Another example is TiddlyWiki - if you define a new widget, or a new filter type for its query language, you do so in JavaScript, and you need to reload the page to see any changes you make take effect.

I really like tight feedback loops, so I wanted to share a little trick I use to shorten those loops for software like this. This technique isn't specific to Awesome or TiddlyWiki - I use it for zsh and urxvt as well, and it can be easily adapted to other pieces of software!

The struggle with these environments is that the painful extension points require you to provide a function or object that implements your extension's behavior, and once that function or object has been provided, references to that value spread throughout the program, and those references are nigh-impossible to change once they've been established. This is similar to early-binding or call-by-reference (as opposed to late-binding or call-by-name) - so the trick is just to use a shim for that early-bound reference that calls out to something that acts like a late-bound reference.

If that sounds straightforward to you, great! Otherwise, here are some concrete examples - hopefully they help the idea make more sense, and perhaps they'll inspire you on how to do this with other pieces of software you use!

Awesome Example

For Awesome, let's say I want to add some behavior around when I dock/undock. The extension point for this is the screen.list signal - it fires every time the list of available screens changes. The early-bound way of doing things looks like this:

screen.connect_signal('list', function()
  print 'screen list has changed'
end)

If I put this in my config file, I have to restart my window manager to see any changes I make. Alternatively, I can put this into a file and pipe it to awesome-client, which lessens the pain somewhat, but if I do that, the old function is still referenced, so I'll end up seeing every version of the code I run this way whenever the screen list changes.

Now, here's the late-bound version:

-- only set up the signal handler once
if not my_late_bound_screen_handler then
  screen.connect_signal('list', function(...)
    return my_late_bound_screen_handler(...)
  end)
end

function my_late_bound_screen_handler()
  print 'screen list has changed'
end

This changes the early-bound handler to look up another function in the globals table by name, so now whenever I change the file I'm working with and update that function via awesome-client, I can see the effect of those changes without needing to restart, without needing to clean up the old handler, and without getting the effects of previous versions. And once I'm all done, I migrate the contents of my_late_bound_screen_handler to its permanent home within my config file and then restart, since at this point I'm (fairly) confident I'm not going to sink my desktop environment 😅

TiddlyWiki example

Here's an example for TiddlyWiki - let's say I'm defining a new widget in JavaScript. Normally, when I make changes I need to reload the wiki entirely, which isn't terrible but presents a longer iteration cycle than I'd like. So instead of this:

// adapted from https://tiddlywiki.com/dev/#Hello%20World%20widget%20tutorial
(function() {

let {widget: Widget} = require('$:/core/modules/widgets/widget.js');

function MyWidget(parseTreeNode, options) {
  this.initialise(parseTreeNode, options);
}

MyWidget.prototype = new Widget();

MyWidget.prototype.render = function(parent, nextSibling) {
  this.parentDomNode = parent;
  let textNode = this.document.createTextNode("Howdy!");
  parent.insertBefore(textNode, nextSibling);
  this.domNodes.push(textNode);
};

exports['my-widget'] = MyWidget;

})();

...I do this:

// in my-widget.js
(function() {

let {widget: Widget} = require('$:/core/modules/widgets/widget.js');

function MyWidget(parseTreeNode, options) {
  this.initialise(parseTreeNode, options);
}

MyWidget.prototype = new Widget();

MyWidget.prototype.render = function(...args) {
  let {render} = eval($tw.wiki.getTiddlerText('my-widget-late.js'));
  return render.call(this, ...args);
};

exports['my-widget'] = MyWidget;

})();

// in my-widget-late.js
(function() {

return {
  render(parent, nextSibling) {
    this.parentDomNode = parent;
    let textNode = this.document.createTextNode("Yo!");
    parent.insertBefore(textNode, nextSibling);
    this.domNodes.push(textNode);
  }
};

})()

...and similar to my Awesome example above, once I'm done, I move what I was working on in my-widget-late.js into the main widget file.