A plugin system?

Being left to only JavaScript built-ins kind of sucks. We know how much work it takes to write a code editor, and definitely don't want to put it on our massive pile of side-projects. Using a raw textarea for writing articles gets exhausting very fast.

We mentioned the CodeMirror editor potentially being a great addition to our project in the past few articles. Let's see what we can do about it.

Dependency trees

CodeMirror in its lightweight version still comprises tens of thousands of lines of JavaScript, and it is split into several modules using ECMAScript import statements. Because these module requirements are not a flat hierarchy, we cannot trivially rewrite these import statements using the magic TypeScript compiler.

The bundling step is pretty much unavoidable if we want to use something as complex as an embeddable text editor. We've tried a bunch of different approaches this week and didn't really get anywhere. Even if we made some progress, we would find ourselves in a situation where we had to commit a large chunk of generated code into the repository.

So, what if, instead of avoiding it entirely, we cheated a little and used a little shim to call into external hooks if they are present? That way, the interface would gracefully fall-back to a basic textarea if a plugin is not present, and replaced it with something more sophisticated if a hook is present.

This idea was not revealed to us in a dream, we had the (mis)fortune of setting up MediaWiki last month.

Reasonable enough, let's try it.

Keeping it simple

The basic idea is to have a user-provided JSON file, which maps event names to an array of user-defined handlers. Having a static configuration file without dynamic plugin discovery allows us to be explicit about when each part gets called and how. This allows us to write quick ad-hoc hooks without going through some complex system of plugin lifecycles.

In a separate project (with npm), we can write the desired hook mounting the editor into the provided HTML element.

// src/index.js
import {EditorView, basicSetup} from "codemirror";
import {markdown} from "@codemirror/lang-markdown";
import {oneDark} from "@codemirror/theme-one-dark";

export default (mount, content) => {  
  const view = new EditorView({
    parent: mount,
    doc: content,
    extensions: [basicSetup, markdown(), oneDark]
  });

  return {
    handle: view,
    getContent: () => view.state.doc.toString()
  };  
}

Now the part where we cheat: we use Rollup to bundle the entire CodeMirror into a single JavaScript file exporting only a single function. The Node resolver extension handles all the dirty work for us and we only need to do this once.

// rollup.config.js
import { nodeResolve } from '@rollup/plugin-node-resolve';

export default {
  input: 'src/index.js',
  output: {
    dir: 'output',
  },
  plugins: [nodeResolve()]
};

The resulting single-JavaScript-file bundle can be uploaded to the web server and does not require any further action unless some Web API changes, the hook contract changes, or the universe reaches heat death.

Dynamically loading modules

We do not have to fetch the configuration JSON as modern ES imports support directly importing JSON files. This feature is somewhat non-standard, but browsers seem to handle it just fine.

import hooks from "/assets/_hooks/hooks.json" with {type: "json"};

There is one potential issue where SWC strips all import attributes during compilation by default. This behavior can luckily be turned off, although it is behind an experimental flag:

{
  "jsc": {
    "experimental": {
      "keepImportAttributes": true
    }
  }
}

Each hook specified in the JSON file requires us to dynamically import the specific file. Nowadays, this is achieved with browser-native dynamic imports, which are also conveniently asynchronous. Although they have the annoying property of putting the default import under "default".

type Hooks = {
    "editor-mount"?: [(mountPoint: HTMLElement, initialContent: string) => ({
        handle: unknown,
        getContent: () => string
    })],
    "editor-unmount"?: [(handle: unknown) => void]
}

const compiledHooks = {} as Hooks;

for (const hook in hooks.hooks) {
    compiledHooks[hook] = [];

    for (const source of hooks.hooks[hook]) {
        const handler = await import(`/assets/_hooks/${source}`);
        compiledHooks[hook].push(handler.default);
    }
}

Going through the object containing all hooks quickly gets tedious though. This is where the shim comes into play. It checks if a handler actually exists, returns null if it does not exist, and executes the corresponding handler if it does. The final function is quite short and we can model certain hooks as only allowing one handler.

type SingleEl<T> = T extends [infer U] ? U : never;

function entryPointOne<T extends keyof Hooks>(hook: T, ...args: Parameters<SingleEl<Hooks[T]>>):
    ReturnType<SingleEl<Hooks[T]>> | null {
    if (!(hook in compiledHooks) || !compiledHooks[hook].length) {
        return null;
    }

    const [hookFn] = compiledHooks[hook] as Hooks[T];
    return hookFn.call(null, ...args);
}

The only part that's left is calling into the shim in desired places. Proper types make this very pleasant to use and the resulting changes are small. It works surprisingly well and requires little to no modifications of the web server.

Also, since the hooks are in the data directory, they can be backed up with all the other dynamic data on this website. The disadvantage is that modifications require us to keep the original project, otherwise we are stuck with an opaque blob that cannot be reproduced.

My two cents

I believe plugin systems driven by demand are much easier to work on, because it is very difficult to tell what should be scriptable before the project even exists. Designing a modular-first system often leads to unnecessary abstraction and developer burnout.

However, any sufficiently old system turns into spaghetti and adding plugin support to an ossified protocol quickly becomes a cat-and-mouse game of keeping the existing codebase and API working.

Hooks provide a compromise between modularity and stability. They can also be opaque and allow exposing private parts of an API without making everything public.